Method and vehicle data system for determining vehicle data for ambient capture sensors

Abstract

Vehicle data system and method for determining useful or communicable vehicle data from ambient capture sensors - Patents.com The present invention relates to a vehicle data system (10) and method for determining vehicle data worth communicating from a surrounding capture sensor (1), the method being employable in the context of improving systems for, for example, assisted or automated driving. The vehicle data system (10) includes a surroundings detection system (16) and a monitor system (15) within the vehicle. The perimeter detection system (16) is adapted to receive and evaluate input data X of the perimeter acquisition sensors (1). The input data X is evaluated by a first trained artificial neural network K, and as a result of this evaluation perimeter detection data Y' is output. The monitor system (15) processes the same input data X of the peripheral acquisition sensor (1) through a second trained artificial neural network K R and outputs the reproduced data X' as a result of this evaluation. If the monitor system (15) detects a difference in the reproduced data X' relative to the input data X that exceeds a threshold, the monitor system (15) prompts the transfer of the input data X to a separate data unit (20). This allows us to focus on scenarios and objects (44) that were not covered thoroughly or at all during the development of the perimeter detection system and, as a result, were not considered as edge cases when training the neural network.

Description

[Technical Field]

[0001] The present invention provides vehicle data of communication value from surrounding capture sensors. Decide For For those law and vehicle data systems Regarding It is something that ,for example, Driving assistance , also is automatic Driving In the context of improving the system Applicable It is possible. [Background technology]

[0002] Conventional technology ,example For example, the data obtained by test vehicles during development Ta Training data to basis Based on this, the network is trained for detection and vehicle functions. of Including This is limited to data from the scenario that occurred at the time. To ensure sufficient coverage of road traffic scenarios, it is advantageous to collect data during the actual use of the vehicle, which allows a wide diversity of data from which to select training data.

[0003] During development, data collection using test vehicles was carried out record Scenarios but However, during the operation of production vehicles, From the scenario of teeth Insufficient or Almost cover And do not have 、 A further scenario arises, which is particularly relevant for edge cases. vinegar( child Here is the boundary case Example, special case Example, individual case For example, or very Rarely occurs raw do Therefore, it is usually a test vehicle. So Not recorded, Also teeth, Very rarely Recorded do not have special caseIn order to develop an AI (Kuenstliche Intelligenz, or AI) that can handle edge cases, it is useful to collect data while using the KI. Transmission In order to reduce the number of data to be processed, data selection within the vehicle is necessary, where the computational load required for data selection must be kept to a minimum due to the limited budget for on-board systems.

[0004] That is, a method that requires only a small amount of computation time to evaluate whether a road traffic scenario is useful for developing KI algorithms for ADAS and AD in the field of mass-produced vehicles would be advantageous.

[0005] A common method for obtaining training data from production vehicles is to Patent Document 1( WO2020 / 056331A1 ) , individual sensors About It is described. car Both include an artificial neural network that evaluates the sensor data. teeth , To determine the classification score for the sensor data, The intermediate results of the neural network Applies At least a portion of the sensor data is transmitted over a computer network based at least in part on the classification score. Transmission In the case of a positive decision, the sensor data is Transmission and used to generate training data. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2020 / 056331 Summary of the Invention [Problem to be solved by the invention]

[0007] SUMMARY OF THE INVENTION It is an object of the present invention to provide an optimized method for efficiently collecting useful data from multiple vehicles. [Means for solving the problem]

[0008] One aspect is Machine learning systems and methods data Based on For the algorithm, or data Driven and Optimized To achieve this, it is important to recognize useful examples and edge cases from the vehicle fleet.

[0009] The vehicle data system of the present invention includes a surroundings detection system and a monitoring system in the vehicle. Preparation do. The perimeter detection system receives and evaluates input data X from the perimeter capture sensors. vinegar The input data X is evaluated by the first trained artificial neural network K, and as a result of this evaluation, peripheral detection data Y' is output. The monitor system synchronizes the peripheral capture sensors. character The input data X is passed through a second trained artificial neural network K R Using As a result of this evaluation, 、 Re construction It is configured to output data X'. The monitor system Exceeding the threshold Replay of input data X construction Data X' deviation If detected (→ potential Edge Case vinegar) , the monitor system receives input data X to another to the data unit Transmission Encourage. Another The data unit may be stored in, for example, an external server, a cloud memory, Also is the backbone of the V2X system. It is okay What is V2X (vehicle-to-X)? ,car Both with other participants General believe communication system orIt means a telematics system.

[0010] In one embodiment, the second artificial neural network K R Ha, oh Toe It is Encoder. Described "Se -Fute Wick Critical It is not" (Monitoring ) system is Toe Encoder By This is done using the detection algorithm described above. or Same input data as the function to basis Developed based on Due to its functional principle, Toe Encoder is , others of possible Compared to the approach There is a big advantage do.

[0011] this is, Sensor Data as Image data of example To, explain 。 picture The statue, oh Toe When entered into the encoder, Auto The encoder is , out In power The image Re Trying to build Try. therefore, Oh Toe Encoder is Additional Label Use without inputting signal Only Use hand training This is related to annotations. Extra work to do but Occurred The advantage is that the law of nature , and furthermore, related to the input data Error of (Even for unknown data) What time is it? Also fixed amount Target decision It is possible advantage In this case, , the appropriate threshold formation is Can O Toe The encoder or Some region against ApplyIt is possible.

[0012] Therefore, Oh Toe Encoder is All Input signal t And, Its own error Machine learning methods typically encounter problems when input data does not match the training data well. Regarding this, Higher error The value is prediction It will be. Toe The encoder and detector are to basis Because it is based , the following advantages are available a R 。 Oh Toe Unknowns identified by the encoder Scenario Or uncertain scenarios These The scenario is the training data 20 Included in minutes This indicates that , In function development Traffic Scenario of Wide stomach range in cover It is important to do.

[0013] According to one embodiment, the monitoring system comprises: deviation Calculate a score that estimates the usefulness of input data X based on

[0014] one Example in is that the first artificial neural network K is predetermined Trained using training data Tei , At that time, For input data X_1, X_2, ..., X_n hand, each the goal Output data Y_1, Y_2, ..., Y_n is Used The first neural network K Weight By adjusting the input data X_1,X_2,...,X_n, the output of the first neural network K is calculated. the goal Output data Y_1, Y_2, ..., Y_n deviation between First to show error The function is minimized.

[0015] According to one embodiment, the second artificial neural network K R is the second artificial neural network K R of Weight By adjusting training difference R, At this time, the replay of the input data X of the peripheral capture sensor construction Data X' deviation The second indicates error The function is minimized.

[0016] one In the embodiment , Zhou In addition to the input data X of the edge capture sensor, meta information but Meta information is transmitted in the following groups: 、 the current Software version, monitor system calculation death Score, GPS data, date, time, vehicle identification number (FI N) , and , on-site of Condition situation Beauty / also is a vehicle of Condition status Cloud data that allows you to understand 、 corresponds to one or more pieces of information from this to basis Based on ,for example, On-site development status of accurate to Can be tracked (simulation using the same software version). emotion Information collection By unknown Scenarios and Select edge cases and vehicle detection function or It can be directly incorporated into the development process of peripheral detection functions. Can This allows Intermittent A quality assurance process can be established. With each development step, more and more useful data is generated. , Shi It can be incorporated into the stem.

[0017] Furthermore, the maturity of the software can be assessed based on the received data. Transmission Number of ByIt is also possible to derive . Forecast measurement of accuracy but No Enough for data Transmission The number of processes few The higher the software maturity, It can be said that .

[0018] According to one embodiment, the perimeter detection system is configured to receive and jointly evaluate input data X from a plurality of perimeter acquisition sensors. This is Mal Chise Nsa of It supports setup. Chise The sensor system can improve the safety of road traffic detection algorithms by validating detections from multiple perimeter capture sensors. There are advantages Maru Chise The sensor system is, for example, 、 - One or more camera, - One or more radar, - One or more ultrasound systems, - One or more Rider of and / or - One or more Microphone Any combination of Configure It is possible.

[0019] this case , Maru Chise A sensor system consists of at least two sensors. One of these sensors but, Time t Recorded in Day Ta is , which can be written as D_(s,t) De Data D is the image and / or also is audio record , and surrounding objects angle, distance, speed degree, Reach Reaction Shooting Measurements such as Yes That's fine. .

[0020] one In the embodiment , Mo The monitor system is configured to process the input data X of the perimeter acquisition sensor in parallel with the perimeter detection system.

[0021] According to one embodiment , Mo The detector system is integrated into the peripheral detection system as an additional detector head. Mo The monitor system and the perimeter detection system use a common encoder.

[0022] one In an embodiment, the input data of the perimeter acquisition sensor is image data. Mo The monitor system plays back the entire image or a partial excerpt of the image. construction This is further configured to Construction Errors Estimate the value of By force Cut.

[0023] According to one embodiment , Mo The monitor system uses the uncertainty scale decision The uncertainty scale is determined by the monitoring system. construction How reliable is it when outputting data X'? What it shows is.

[0024] one Example in teeth , Mo Nita System is construction It is configured to take into account the consistency of data X' over time.

[0025] in this case, In one implementation variation , Mo Renewal of the Nita System construction Data X' from input data X deviation But continuous occurs in or is only available for a limited time arising Will it be but Distinguished.

[0026] one In the embodiment, the perimeter detection system and the monitor system of both but ,wireless in It is configured to be updatable.

[0027] A further object of the present invention is to provide a method for detecting and transmitting valuable vehicle data from surroundings acquisition sensors. Decide The method comprises the steps of: include. Using a trained first artificial neural network 、 Perimeter detection system That Evaluating the peripheral acquisition sensor input data X 、 When evaluating decision outputting the detected peripheral data Y'; 、 The second trained artificial neural network K R Using 、 Monitor system That Peripheral capture sensor character Step to evaluate the input data X 、 When evaluating decision Re construction Step to output data X' 、 and, Replay for input data X that exceeds the threshold construction Data X' Detect deviation Where In this case, input data X is Data Units Transmission reach do instructions of Steps to do this.

[0028] The usefulness of traffic scenarios is evaluated by Chise Nsa of Setup, Transitions, and Networks but Confidence in predicting output degree Further improvements can be achieved by taking into account the estimation of Can Furthermore, the short computation time is advantageous for embedded systems. These points can be calculated, for example, load Low noise monitor system or autoencoder Daa Proach but choice And ,this of , Maru Chise Sensor System For To, trust degree Estimates and time series Na verification for By extension, vs. be punished do.

[0029] The following examples and figures are explained in more detail. 。 [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 shows a vehicle equipped with perimeter acquisition sensors and data systems. [Figure 2] FIG. 2 shows the perimeter acquisition sensors, data system, and other data units. [Figure 3] FIG. 3 shows an example of a classification system and a monitoring system as a perimeter detection system. [Figure 4] Figure 4 shows a vehicle capturing unknown scene conditions / status. [Figure 5] Figure 5 shows a grayscale image from an in-car camera with overexposure due to poor exposure control. [Figure 6] Figure 6 shows a grayscale image from an in-car camera containing artwork that may lead to unpredictable conditions during evaluation. DETAILED DESCRIPTION OF THE INVENTION

[0031] FIG. 1 illustrates a perimeter acquisition sensor 1 and a data system 10. and Vehicle 2 equipped with vinegar . description The surroundings capture sensor 1 is located inside the vehicle 2. ( For example, the rearview mirror area ) placed in Tei , which corresponds to a camera capturing the surroundings of vehicle 2 through the windshield. vinegar Car 2 Another The perimeter acquisition sensor 1 may be, for example, a camera sensor, a radar sensor, a lidar sensor, and / or alsois an ultrasonic sensor It is okay (not shown). All peripheral capture sensors 1 teeth , information about the surroundings of vehicle 2 include These data are transmitted to and processed by the vehicle's data system 10. De The data system is, for example, Driving assistance that derives detected objects from sensor data and understands the vehicle's surroundings and traffic conditions to some extent , or automatic Driving (ADAS or AD, e.g., Autonomous Driving Control Unit ADCU) of Control device for Such control devices May contain Based on these findings, Driving assistance Now, driver to caveat or , To vehicle control limited Na intervention is carried out , automatic driving So, ADCU is vehicle 2 whole The regulation Conduct the administration It is possible to do so.

[0032] FIG. 2 shows a peripheral acquisition sensor 1, a data system 10, and another Data unit 20 vinegar .

[0033] De Data System 10 , small At least one peripheral acquisition sensor 1, e.g., image capture Device and , in car 2 Electrically connected . picture Photographing Device is the front camera of the vehicle. It may be possible. The front camera is of ahead a surrounding area of capture do Acts as a sensor .centre Front camera signal or Based on the image data, the surrounding area of ​​vehicle 2 of detection can be These surrounding detection objects to basis Zui For example, lane recognition, lane keeping support, road sign recognition, speed limit assistance strike , Traffic participant recognition, Collision warning, Emergency braking assistance strike , vehicle distance tracking control, construction site assistance strike , Out Bar Npa Ilot (highway autopilot), Cruising Good luck Hand function, and / or Also ADAS or AD functions such as autopilot are provided by the ADAS / AD control unit. can be do. Image capture Device is typically an optical system or Lens and image capture sensor, e.g., CMOS sensor Preparation do.

[0034] Data captured by Periphery Capture Sensor 1 or data signal The data is transmitted to an input interface 12 of the data system 10. The data is processed within the data system 10 by a data processor 14. De The data processor 14 controls the perimeter detection system 16 and Monitor System 15 Including. The edge detection system 16 may include a first artificial neural network, e.g., a CNN. It may contain The edge detection system 16, in addition to pure detection, More comprehensive Perspective and situation Understanding behavior For example, the surroundings detection system 16 can generate a trajectory prediction for the vehicle 2 itself, as well as trajectory predictions for other objects or traffic participants in the vicinity of the vehicle 2. ,car Both ADAS systems or AD System action It may be safety-critical because of the dependencies on it and warnings. , Mo The monitor system 15 has the main task of detecting the surroundings of the surrounding detection system 16. of Monitoring And Data Another Decide whether to transmit to the data unit 20 fixed It is to For , not safety critical Mo The monitor system 15 is a second artificial neural network, e.g. Toe Encoder May contain . This means thatThe data system 10 or data processor 14 is a single processor for the artificial neural network so that the artificial neural network can process the data in real time within the vehicle. End Hardware accelerator of May contain .

[0035] The detection of the monitor system 15 is 、 For example, exceeding a threshold Detection deviation , the data system 10 outputs via the output interface 18: Data to another Data Unit 20 (Cloud, Backbone, Infrastructure...) Hemu line in Transmission To do The following will be implemented.

[0036] Figure 3 shows the distribution of the peripheral detection system. kind 1 illustrates an embodiment of a system and a monitor system.

[0037] minutes kind The system K classifies the object based on the sensor data X of the peripheral capture sensor 1, for example. kind Minutes kind In addition to system K, an independent and additional second monitor system K R will also be introduced.

[0038] Figure 3a shows the machine learning Using minutes kind System K and monitor system training vinegar How to Examples of the law of flowchart to Show vinegar . training Meanwhile, both systems character By sensor data X training will be done. Classification system K( For example, decision tree learning systems, support vector machines, learning systems based on regression analysis, Bayesian Nne network, neural network, or convolutional neural network etc.) of training In order to training For input data X and training The target value Y is offer Minutes kind Using System K, training Output data Y' is generated from input data X for the monitor system K. R (Oh Toe By using training For input data X, training For input data X, similar did Re construction Data X' Made by It is made. kind System K training The goal is to Without over-adapting Output data Y' Training Target value Y to , as much as possible Get closer This is what we should do. For this purpose, The created output data Y' and training From the target value Y, error Using the function Loss, output data Y' and training Between the target value Y and Deviations that still exist but decision This will be deviation of For example, backpropagation The law Through, minutes kind It is used to adapt the parameters of the system K. predetermined one Match be achieved mosquito , or over Overadaptation This is repeated until signs of Monitor System K R Ha, oh Toe Encoder to basis Zui Therefore, no further annotation is required other than the actual sensor data X. R of training The goal is to construction Data X' Training Input data for X to As much as possible Get closer The created reproduction construction Data X' and training For input data X, the second error Using the function Loss2, construction Data X' and training Between input data X and Still exists R deviation but decision will be done. This deviation For example, monitor system K via backpropagation R This is used to adapt the parameters of predetermined one Match be achieved mosquito , or over Overadaptation This is repeated until the symptoms appear. R of training For example, kind System K training later can be done , At that time, same character Input data X of use That it is necessary Note death It must be. Oh Toe Encoder or Monitor System K R always outputs Former can be compared with sensor data and quantified (using metrics) Error or The quantified uncertainty U can be calculated. construction Data X' and Input data X deviation of is determined by the metric decision In this way decision was deviation The value quantifies the uncertainty U of the output data Y'. transformation do.

[0039] FIG. 3b is an example of the embodiment of FIG. conforms to, minutes kind System K and Monitor System K R of suitable 1 shows a flowchart of a method for using vinegar .this teeth , minutes kind Uncertainty U of output data Y' of system K of quantitative To become This makes it possible. This field The theory is Nita System KR Itself, In application (inference), Metrics Use and the input signal or Replay of input data X construction Data X' error can be measured at any time advantage There is.

[0040] minutes kind System K too , Monitor System K R Also, character Developed based on data are Therefore, both systems Similar It has a defect. kind For system K, the defect in output data Y' is So Unable to measure So , Monitor System K R From the relationship Special fixed vinegar It is possible.

[0041] For example, input data X is image data of an image, and output data Y' is a component corresponding to an object depicted on the image. kind data Tosu R Mo Nita System K R Using of Similar to image X construction If image X' can be created, monitor system K R and minutes kind System K training Sometimes , kind Similar input data X is already This suggests that there was a significant difference between the two, and therefore the uncertainty U of the output data Y' is low. construction Image X' is the original of If the image is significantly different from image X, monitor system K R and minutes kind System K training Sometimes it suggests that similar input data X was not used, and therefore the uncertainty U of the output data Y' is large. Large uncertainties U in the input data orThe input data suggests that this was an edge case.

[0042] Figure 4 shows the unknown Situation on the ground / condition status Vehicle 2 that captured Overview Targeted vinegar An unusual object 42, in this case an elephant, is present in the capture area 44 of the vehicle's surrounding capture sensors. training of the system Applicable In training For data teeth extraordinary Na Object 42 did not include an elephant. to , during development Reproduction The situation was not status This will lead to limit target Na Condition status Monitor System K R By identification Can, this is Exclamation mark in the symbol 46 in Figure 4 Or Caution! Therefore, vinegar The data acquired by the perimeter acquisition sensors can then be used to further develop the detection system K. For this purpose, the data can be used, for example, in telematics Device via an external data unit, e.g., a server or Wireless to Cloud 50 in Transmission 48 vinegar Within the scope of possible development processes, data stored in the cloud can be used for future optimized training Also using this borderline cases It is used so that you can be prepared for such situations.

[0043] According to this system, training The data is sufficient Multiple unrepresented Day Ta be specified. Not represented The data is specific Driving status status This is partly due to the limitations of the peripheral detection sensors in the area, but it is also due to the fact that the surrounding area is simply unusual. ofIt may be due to the scenario.

[0044] Figure 5 is , dew Light Control Problems with Excessive exposure caused by There is an in-vehicle camera Grayscale images In summary In brief vinegar .child The dawn Too much hand For images with low contrast kind There are many case In this situation, status teeth, Due to limitations in exposure control, For example, when you come out of a tunnel, Under similar driving conditions This happens frequently and is due to the nature of the sensor.

[0045] Figure 6 shows the road of safe place On the obi Schematic representation of a grayscale image depicting the unexpected effects of modern art. vinegar The artwork shown here is from the UK , Trafalgar Way, Blackwall, London, 5T G This is a "Traffic Light Tree." This modern artwork consists of a number of traffic lights that do not serve to control traffic. Camera-based traffic light recognition would be unable to attribute traffic control information to this artwork.

[0046] Further aspects and embodiments are described below. 。

[0047] Driving assistance From Ritual Transitioning to autonomous driving is a technical hurdle. Autonomous systems can sometimes be difficult to navigate through complex scenarios that cannot be fully covered by simulations or test drives. Moko Why It is necessary The goal here is to ensure that peripheral capture for vehicle functions is always possible and in as many scenarios as possible. ( Ideally, all scenarios ) In surely It is to function.

[0048] To solve this problem, we will analyze this complex and useful data. ,fruit From road traffic For development Additional monitoring systems that automatically collect Wick Propose a system that is not critical 。「 "Useful" data is software version Every Because it can change to , Shi The stem is updated Can , and raw data In addition , version control Also equipped need Yeah R . stem whole The details are as follows: 。

[0049] -fruit outfit should be , or monitoring should be Vehicle functions are the same character data to basis Made Independent second monitor system About It will be expanded. Therefore, Nita System teeth , vehicle functions When to apply , as well as Indications will be done.

[0050] - During operation, Nita System , Shi Data covered by the system In relation to , the current situation status Cover Range of of Continued Continuous monitoring is performed. For example, threshold control is sufficient. Na deviation but arising If so, the data but Captured for development and passed on to further development of vehicle functionality. child For , Mo Nita System is one End Sen Sa Therefore record was On-site situation whole, Also teeth, Some of the situation on the ground, For example, individual objects ToIt can be evaluated.

[0051] -B In addition to this data Tapa Package to teeth, the current The software version of Usefulness of the data The monitor system calculates Tasu Core, GPS information, day With , FIN (car both identification numbers), Also is connected Dobi Work Luk Lau Dode - Tana Any meta information Included This to basis Based on , On-site development status of accurate to Can be tracked (simulation using the same software version). emotion Information collection By unknown Scenarios and Edge cases can be selected and directly incorporated into the development process of vehicle functions. Can This allows Intermittent A quality assurance process can be established. With each development step, more and more useful data is generated. , Shi It can be incorporated into the stem.

[0052] -More for , receiving Data to be transmitted Transmission Number of The maturity of the software also depends on to derive but It is possible.

[0053] Extending to multi-sensor / multi-network setups

[0054] The proposed monitor system here comprises: Driving assistance and Ritual In the context of the Sase This is an optional in , Maru Chise hmm Sase It is extensible to setups. Chise The sensor system is a Sa By verifying the detection by the system, the safety of detection algorithms for road traffic can be improved. There are advantages Maru Chise The sensor system is, for example, 、 - One or more camera, - One or more radar, - One or more ultrasound systems, - One or more Rider of and / or - One or more Microphone Any combination of Configure It is possible.

[0055] At this time, Mar Chise A sensor system consists of at least two sensors. The data acquisition of one of these sensors s at time t is hereafter denoted as D_(s,t). this Data D is the image and / or Also is an audio recording, and objects in the periphery angle, distance, speed , and reflection Measurements of Yes That's fine. .

[0056] How it works

[0057] The above Monitor system ( or "Safety Wick "Non-critical systems" are Toe Encoder By fruit current This is because subject Detection algorithm or Based on the same data as the function Developed based on O Toe The encoder can be implemented in a variety of ways. , Examination Output Algorithm In addition to the autoencoder in parallel Use Another example is the implementation of an additional detector head. current do This is , i.e., detection for and Oh Toe One method is to use a common encoder for the encoder. Also, The following systems Use as too Think , In other words A monitoring system that starts after the perimeter detection is performed is also considered. Be do. Its functional principle is Toe Encoder is , others It offers significant advantages over the previous approach. We will explain it using an example of image data. 。

[0058] One image is Toe When entered into the encoder, Auto The encoder is Image of Reproduced in the output construction Try to do so. therefore, Oh Toe Encoder is Additional Without labels, using only the input signal training This is possible. on the one hand, Annotation This creates additional work for The advantage is that the law of nature , On the other hand, Related to input data do Error (Even for unknown data) What time is it? Also fixed amount Decided on The advantage is that it can a do. in this case , appropriate thresholding is recommended. Toe The encoder is used to Also is part of region against Apply It is possible.

[0059] Therefore, Oh Toe Encoder is All The input signal t And, Its own error Machine learning methods typically encounter problems when input data does not match the training data well. Regarding this, Higher error The value is prediction It will be. Toe The encoder and detector are to basis Because it is based , the following advantages are available a R 。 Oh Toe Unknowns identified by the encoder Scenario Or uncertain scenarios These The scenario is the training data 20 Included in minutes This indicates that ,traffic scenarios of Wide stomach range in cover To do However, it is useful for function development. do.

[0060] Uncertainty Scale of Expansion

[0061] moreover, Network Re Construction certain fruit Gender estimation Toe In addition to the encoder output ,So Based on this Toe The coder is the one making the decision. fruit Measures of the degree of certainty are also useful. fruit The gender scale is Toe Encoder of Complementing the output, especially the overhaul related to various sensors Toe Encoder of Output fusion If you do, and / Also Ha, oh Toe Encoder of output of Over time to Melt Combine R case It is useful for Such certainty fruit The sex scale is based on statistical calibration or uncertainty. of Estimate By The following are suitable for this: a) The output of the autoencoder is Probabilistic Na Uncertainty table As Weighting do , statistical calibrationThis method is suitable when computational resources are very scarce. b) Enough Computational resources Available If the model of Estimating Uncertainty To be able to For example, Monte Carlo Rod Dropout By Extending each network layer It is possible with This allows you to do the following at runtime: this The network layer and subsequent network layers are pieces Randomized activation of individual neural layers together , m times, resulting in a set of outputs y_i (i=1...m). c) Further measurements of Uncertainty of Estimate can be R 。 for example, error Function (Loss; Loss2) is measured at runtime. of By adding regularization to measure uncertainty It can be achieved . Uncertainty Estimation by Unknown Situation on the ground or uncertain Situation on the ground of Decide The extension for kind The means were decided correctly, but there seems to be a great deal of uncertainty about the decision. Situation on the ground It allows for the evaluation of uncertainty. of By making an estimate, the unknown Situation on the ground Inspection rope Robustness , and search for uncertain on-site conditions It will be expanded. therefore, child NoA architecture allows for the unknown The situation on the ground Not only that, but the network is uncertain. Situation on the ground You can also select do.

[0062] Over time Na one Consistent Expansion

[0063] The above like , Oh Toe Encoder is trainingEnough data for use Not reflected This principle provides a way to identify data over time. Na Further value can be gained by combining this with the idea of ​​consistency. For example, Over time Dynamically filter and continuously generate sample data and Sporadic Distinguishing between outliers that occur only occasionally provides valuable additional information. Short-term abnormal values ​​may suggest sensor-related factors, for example. I'm In short, when entering a tunnel, the white balance of the camera may change significantly at a certain time t. this The data will be useful for the development of each sensor. If the identified data appears consecutively, this means: Useful for algorithm development It's an unknown scenario possibility High stomach .

[0064] From the above-described embodiments and aspects, the following There are advantages. -Special advantage : Run time optimization To do - Any sensor data and their combinations suitable available Being -Development process Easy integration into , Individual Data collection teeth Unnecessary Being -Comprehensive coverage compared to event-based approaches that only monitor the output of vehicle functions, e.g., object track loss To do - consideration difference can algorithm Especially for useful Na Automatically select data To do -re construction Identify realistic scenarios that are difficult to identify (accidents, low probability combinations of events...) To do The present application relates to the invention described in the claims, but also includes the following as other aspects. 1. A surrounding detection system (16) and a monitoring system (15) are installed in a vehicle (2). Preparation A vehicle data system (10) comprising: The aforementioned Perimeter Detection System(16) teeth , receives input data X from the periphery capture sensor (1), evaluates it using a trained first artificial neural network K, and outputs periphery detection data Y' as a result of this evaluation. vinegar It is configured so that The aforementioned Monitor System(15) teeth , the same of the peripheral capture sensor (1) character The input data X is passed through a second trained artificial neural network K R Using and evaluate it. child of Evaluation As a result, construction Output data X' vinegar It is configured so that 、 The aforementioned The monitor system Exceeding the threshold For input data X and reconstruct Data X' deviation If detected, The aforementioned The monitor system (15) receives input data X to another Data Unit (20) Transfer sending instruct to do so, A vehicle data system comprising: 2. The aforementioned Second Artificial Neural Network K R But, oh Toe The vehicle data system (10) according to claim 1 is an encoder. 3. The aforementioned Monitor System(15) teeth , deviation 3. The vehicle data system (10) according to claim 1 or 2, wherein a score for estimating the usefulness of the input data X is calculated based on the above. 4. The aforementioned First Artificial Neural Network K teeth , predetermined Trained using training data Tei , For input data X_1, X_2, . . . , X_n, respectively, the goal Output data Y_1, Y_2,..., Y_n is Used and the first neural network K Weight By adjusting the input data X_1,X_2,...,X_n, the output of the first neural network K is calculated. the goal Output data Y_1, Y_2, . . ., Y_n deviation between First to show error The function (Loss) is minimized R 4. A vehicle data system (10) according to any one of 1 to 3 above. 5. Second Artificial Neural Network K R However, the second artificial neural network K R of Weight By adjusting training And, The aforementioned Replay of input data X from the peripheral capture sensor (1) construction Data X' deviation The second indicates error The function (Loss2) is minimized R 5. A vehicle data system (10) according to any one of 1 to 4 above. 6. In addition to the input data X of the peripheral capture sensor (1), Te, Me Information but transmitted, The aforementioned Meta information teeth , the following groups 、 the current Software version, monitor system but calculation death Score, GPS data, date, time, vehicle identification number No., and , scene situation situation Beauty / also vehicle status Tracking Cloud data that allows One or more pieces of information from6. A vehicle data system (10) according to any one of 1 to 5 above, characterized in that it is compatible with the above. 7. The aforementioned Perimeter Detection System(16) teeth , multiple The aforementioned The vehicle data system (10) according to any one of 1 to 6 above, characterized in that it is configured to receive input data X from the surroundings capturing sensors (1) and evaluate them all at once. 8. The aforementioned Monitor System(15) teeth , The aforementioned The input data X of the peripheral capture sensor (1) is The aforementioned 8. The vehicle data system (10) according to any one of items 1 to 7 above, characterized in that it is configured to process in parallel with the surroundings detection system (16). 9. The aforementioned Monitor System(15) teeth , as an additional detection head The aforementioned Incorporated into a perimeter detection system (16); and The aforementioned Monitor system (15) and The aforementioned Perimeter Detection System(16) teeth 9. A vehicle data system (10) according to any one of items 1 to 8 above, characterized in that a common encoder is used. 10. The aforementioned Periphery capture sensor (1) input data teeth image data, and The aforementioned Monitor System(15) teeth , the whole image or a partial excerpt of the image construction 10. A vehicle data system (10) according to any one of items 1 to 9 above, characterized in that it is configured to: 11. The aforementioned Monitor System(15) teeth , the uncertainty measure decision death, and It is configured to output And, The aforementioned Uncertainty Scale teeth, monitor system(15) of Re construction When outputting data X' The monitor system (15) How certain is Or On a scale showing The vehicle data system (10) according to any one of the above items 1 to 10, characterized in that: 12. The aforementioned Monitor System(15) teeth , re construction 12. A vehicle data system (10) according to any one of 1 to 11 above, characterized in that it is configured to take into consideration the consistency of the data X' over time. 13. The aforementioned Re-use of monitor system (15) construction Data X' from input data X deviation But continuous arise in Ruka, Also is only available for a limited time arise Or of distinction vinegar 13. The vehicle data system according to claim 12, 14. The aforementioned Perimeter detection system (16) and The aforementioned Both the monitor system (15) Line A 14. The vehicle data system according to any one of 1 to 13 above, characterized in that it is configured to be updateable. 15. Zhou Edge capture sensors (1) transmit valuable vehicle data Decide method And, The method comprises the steps of: Using a trained first artificial neural network 、 Perimeter detection system (16) That Evaluating the input data X of the perimeter acquisition sensor (1) 、 When evaluating decision outputting the detected peripheral data Y'; 、 The second trained artificial neural network K R Using 、Monitor system (15) That Evaluating the same input data X of the peripheral acquisition sensor (1) 、 When evaluating decision Re construction Step to output data X' 、 and, For input data X that exceeds the threshold, do Re construction Data X' Detect deviation Where If, enter Force Data X to another data unit (20) Transmission To Instruction steps 、 A method including .

Claims

1. A vehicle data system (10) comprising a surroundings detection system (16) and a monitoring system (15) in a vehicle (2), The monitor system (15) is incorporated into the periphery detection system (16) as an additional detection head, and the monitor system (15) and the periphery detection system (16) use a common encoder; the perimeter detection system (16) is configured to receive input data X from the perimeter acquisition sensor (1), evaluate it using a trained first artificial neural network K, and output perimeter detection data Y' as a result of this evaluation; The monitoring system (15) applies the same input data X of the peripheral acquisition sensor (1) to a second trained artificial neural network K Ｒ and outputting the reconstructed data X' as a result of the evaluation, If the monitoring system detects a deviation of the reconstructed data X' from the input data X that exceeds a threshold, the monitoring system (15) instructs the transmission of the input data X to another data unit (20). A vehicle data system comprising:

2. Second Artificial Neural Network K Ｒ 2. The vehicle data system (10) of claim 1, wherein: is an autoencoder.

3. The vehicle data system (10) of claim 1 or 2, characterized in that the monitor system (15) calculates a score that estimates the usefulness of the input data X based on the deviation.

4. The first artificial neural network K is trained using predetermined training data; 3. The vehicle data system (10) of claim 1, wherein target output data Y_1, Y_2, ..., Y_n are used for input data X_1, X_2, ..., X_n, respectively, and a first error function (Loss) indicating the deviation between the output of the first neural network K for the input data X_1, X_2, ..., X_n and the corresponding target output data Y_1, Y_2, ..., Y_n is minimized by adjusting the weights of the first neural network K.

5. Second Artificial Neural Network K Ｒ However, the second artificial neural network K Ｒ It is trained by adjusting the weights of A second error function (Loss ２ 3. The vehicle data system (10) of claim 1 or 2, wherein:

6. In addition to the input data X of the perimeter acquisition sensor (1), meta-information is transmitted, said meta-information comprising the following groups: the current software version, a score calculated by the monitoring system (15) estimating the usefulness of the input data X, GPS data, date, time, vehicle identification number, and cloud data allowing to track the site and / or vehicle conditions; 4. The vehicle data system (10) of claim 3, wherein the vehicle data system (10) corresponds to one or more pieces of information from:

7. A vehicle data system (10) as described in claim 1 or 2, characterized in that the surrounding detection system (16) is configured to receive input data X from multiple surrounding capture sensors (1) and evaluate them collectively.

8. A vehicle data system (10) as described in claim 1 or 2, characterized in that the monitor system (15) is configured to process the input data X of the periphery capture sensor (1) in parallel with the periphery detection system (16).

9. A vehicle data system (10) as described in claim 1 or 2, characterized in that the input data of the peripheral capture sensor (1) is image data, and the monitor system (15) is configured to reconstruct the entire image or an excerpt of a portion of the image.

10. The monitoring system (15) is configured to determine and output an uncertainty measure; 3. The vehicle data system (10) of claim 1 or 2, wherein the uncertainty measure is a measure indicating how certain the monitoring system (15) is when outputting the reconstruction data X' of the monitoring system (15).

11. A vehicle data system (10) as described in claim 1 or 2, characterized in that the monitor system (15) is configured to take into account the consistency of the reconstructed data X' over time.

12. A vehicle data system as described in claim 11, characterized in that it distinguishes whether the deviation of the reconstructed data X' of the monitor system (15) from the input data X occurs continuously or only for a limited period of time.

13. A vehicle data system as described in claim 1 or 2, characterized in that both the surrounding detection system (16) and the monitor system (15) are configured to be able to be updated wirelessly.

14. A method for determining vehicle data worth transmitting from a perimeter capture sensor (1) using a vehicle data system comprising a perimeter detection system (16) and a monitor system (15) within a vehicle (2), comprising: The monitor system (15) is incorporated into the periphery detection system (16) as an additional detection head, and the monitor system (15) and the periphery detection system (16) use a common encoder; The method comprises the steps of: evaluating the input data X of the perimeter acquisition sensor (1) by said perimeter detection system (16) using a trained first artificial neural network; outputting the perimeter detection data Y' determined during the evaluation; Trained second artificial neural network K Ｒ and evaluating the same input data X of the perimeter acquisition sensor (1) by the monitoring system (15), outputting the reconstruction data X' determined during the evaluation; and Instructing the transmission of the input data X to another data unit (20) when detecting a deviation of the reconstructed data X' with respect to the input data X that exceeds a threshold; A method comprising:

Images