Method for detecting time-critical events in the real world using a sensor

Abstract

A method is proposed for capturing time-critical events in the real world using a sensor, wherein a timestamp is set at a point in time at the end of the captured event, at which the data captured by the sensor has been converted into a machine-readable, electronically evaluable data format, and another timestamp is set at a point in time before the start of the capture by the sensor, and an average of the two times in time is calculated, which is set as the timestamp for the time at which the event took place.

Description

[0001] The present invention relates to the field of detecting time-critical events using a sensor.

[0002] Accurately capturing time-critical events in the real world using sensors is crucial for many applications. Particularly in the field of (at least partially) autonomous driving, precise environmental perception and the recording of real-world events are essential. Equally important is the accurate recording of real-world events in industrial applications such as assembly processes, chemical processes, and other processes where precise, time-critical event detection is required. Increasingly, applications require the simultaneous time-stamping of data from multiple sensors; that is, several sensors capture the same event in the real world, for example, from different perspectives or using different technologies, with each sensor's data being time-stamped.To better describe the same real-world event using data from multiple sensors, such data with the same or closely spaced timestamps are combined, a process also known as sensor fusion. Accurate timestamps are essential for this process as well.

[0003] Currently, the data captured by a sensor is read and converted into a machine-readable, electronically analyzable data format, either directly in a processing unit within the sensor or in an external processing unit. After this processing, i.e., when electronically analyzable data is available, a timestamp is set to represent the recording of the event. Due to this process, there is always a time delay between the actual event recorded by the sensor and the time at which the timestamp was set.

[0004] Since the most accurate possible recording of the time (or period) of time-critical events in the real world is important in many applications, the invention is based on the objective of providing an optimized method for recording time-critical events in the real world using a sensor.

[0005] This task is solved by the features of independent claims. Advantageous embodiments are the subject of dependent claims.

[0006] A method is proposed for capturing time-critical events in the real world using a sensor, wherein a timestamp is set at a point in time at the end of the captured event, at which the data captured by the sensor has been converted into a machine-readable, electronically evaluable data format, and another timestamp is set at a point in time before the start of the capture by the sensor, and an average of the two times in time is calculated, which is set as the mean time at which the event took place.

[0007] In one implementation, an initial processing time for reading, processing, and storing the subsequent timestamp is added to the time before the sensor began recording.

[0008] In one implementation, the first processing time is known or defined as a specific time period, or a probability is determined for a range of values ​​into which the first processing time falls, based on previous measurements of processing times and statistical procedures.

[0009] In one implementation, a second processing time between the actual end of the recording and the provision of the machine-readable, electronically evaluable data format is subtracted from the timestamp of a point in time at the end of the recorded event.

[0010] In one implementation, the second processing time is known or defined as a specific time period, or a probability is determined for a range of values ​​into which the second processing time falls, based on previous measurements of processing times and statistical procedures.

[0011] Furthermore, a computer program is provided that is set up to execute individual or all steps of the procedure using program code on a computing unit.

[0012] Furthermore, a computing unit is provided on which the computer program is implemented.

[0013] In one version, the computing unit is part of an in-vehicle computing unit of a vehicle that is at least partially automated, or part of an industrial processing plant.

[0014] Further features and advantages of the invention will become apparent from the following description of exemplary embodiments of the invention, with reference to the figures of the drawing, which illustrate details of the invention, and from the claims. The individual features can be implemented individually or in combination in a variant of the invention.

[0015] Preferred embodiments of the invention are explained in more detail below with reference to the accompanying drawing.

[0016] Fig. Figure 1 shows a basic process flow according to one embodiment of the present invention.

[0017] As already mentioned, accurately recording the time (or period) of time-critical events in the real world (as opposed to an event in the digital world) is important for many applications.

[0018] For example, a real-world event is the detection of light rays by an imaging sensor, e.g., to capture a camera image. This detection can occur at different times. Typically, for example, in a single event (in the real world), the functioning cells of a CCD sensor (detection area 1) are not exposed to the light rays simultaneously if a focal-plane shutter 2 is used that moves across the cells, as in Fig. Figure 1 shows. In this case, it is not clear which time point should be defined as the time at which the light rays reached the CCD cells if the resolution of the timestamp information is greater than the opening time of the focal-plane shutter 2 at a particular cell.

[0019] To optimize the determination of the time of recording an event in the real world using a sensor, in addition to the timestamp t already used in the prior art, tsa A second timestamp is added at the end of the event recording. tsb introduced as in Fig. 1 shown using the above described design of the CCD sensor with slotted shutter 2.

[0020] Fig. Figure 1 shows a time sequence (time axis t) of an event being detected by a CCD sensor with a slotted shutter 2, which moves over the detection area 1 of the CCD sensor for a specific time. Furthermore, the time t startexp of the actual start of the recording and the time t endtexp The actual end of the event detection by the sensor is plotted. These times t startexp and t endtexpHowever, these values ​​are either unknown, unavailable, or not passed to the processing unit. Furthermore, the timestamp t is also unknown. tsa The end of the acquisition process is shown, which is already used in the prior art to determine the time of the event. At this point, the acquired sensor data has already been processed in a processing unit within the sensor or an external processing unit and converted into a machine-readable, electronically processable data format.

[0021] To optimize the determination of the time of the event, a further timestamp t is used according to the invention. tsb introduced, which occurs before the sensor begins to detect the event, i.e., before time t startexp , is determined. Since each sensor requires a signal to start a detection cycle, the exact time t is not fixed. startexpThe actual start time of the recording is unknown, but a timestamp t can be recorded before the signal to start recording is sent. tsa be set, which will then in any case be before time t startexp lies.

[0022] By introducing the timestamp t tsb in addition to the timestamp t tsa can be derived from the two time points, which are defined by both timestamps t tsa , t tsb The time average (time t) is defined as the time average. mean ) can be calculated. This time t mean This is defined as the mean time at which the event occurred. This allows for a more precise determination of the time of the event than before.

[0023] The timestamp t tsb is just before time t startexp of the actual start, but will probably not be at time t startexpof the actual start of data acquisition, since there is always a processing time D1, albeit minimal, in the form of a time delay between reading, processing (e.g. conversion) and saving the timestamp t tsb until the physical sensor information is present or required.

[0024] The same applies to the timestamp t tsa at the end of the data acquisition. Here too, a processing time D2 exists between time t. endtexp the actual end of the data collection and the provision of the machine-readable, electronically evaluable data format is present.

[0025] To further optimize the determination of the event time, both processing durations D1 and D2 are preferably included in the calculation of the mean value to determine time t. meanof the event. The processing duration D1, D2 can be known or defined as a specific time interval. The values ​​of the processing duration D1, D2 are used accordingly for the calculation of t. startexp and t endtexp used. The following applies: t startexp = t tsb + D1 and t endexp = t tsa - D2.

[0026] If the processing durations D1 and D2 cannot be precisely determined, then the tolerance range for determining the real-world event time is larger than if the processing durations D1 and D2 were known. In this case, prior separate measurements of the processing durations D1 and D2, combined with statistical methods, can narrow down the probability that these durations fall within a specific range, thus allowing for at least a more accurate prediction of the timestamp. This, in turn, allows for a statement regarding the probability that a real-world event occurred at or near the given timestamp.

[0027] By introducing the second timestamp t tsbBefore the sensor begins recording the event, a more accurate prediction of when the event occurred can be made than before. Further optimization is achieved by incorporating known time delays due to processing durations D1 and D2.

[0028] Sensors that can detect time-critical events include imaging sensors such as CCD sensors, but also radar sensors, lidar sensors and acoustic sensors.

[0029] Real-world applications include at least semi-automated to autonomous driving applications, particularly object and environment perception, as well as industrial applications where processes must run simultaneously or in a very precisely timed sequence, such as assembly or chemical processes. The described method can also be used in sensor fusion applications to enable better temporal recording of monitored events.

[0030] The process is implemented as a computer program on a processing unit. This unit is, for example, part of an in-vehicle computing unit in a vehicle with at least partial automation, or part of an industrial processing plant. The computing unit can also be the sensor itself if it has a corresponding processing unit. Reference symbol list 1 Sensor detection range 2 slot closure t time t tsa Timestamp End of recording t tsb Timestamp Start of recording t startexp Time of actual start of recording t endtexp Time of actual end of recording t mean calculated mean time of the event D1 Processing time before event D2 Processing time after event

Claims

[1] Method for detecting time-critical events in the real world by means of a sensor, wherein - a timestamp (t tsa ) is determined at a time at the end of the recorded event at which the data recorded by the sensor are converted into a machine-readable, electronically analyzable data format, and - another timestamp (t tsb1 ) is determined at a time before the sensor begins detecting, and - an average of the two points in time is calculated, which is called the mean point in time (t mean ) on which the event took place. [2] Method according to claim 1, wherein a first processing time (D1) for reading, processing and storing the further time stamp (t sb ) is added to the time before the sensor starts detecting. [3] Method according to claim 2, wherein the first processing time (D1) - is known or determined as a specific period of time, or - based on previous measurements of processing times and statistical methods, a probability is determined for a range of values ​​into which the first processing time (D1) falls. [4] Method according to claim 1, 2 or 3, wherein a second processing period (D2) is between the time (t endtexp ) of the actual end of recording and the provision of the machine-readable, electronically analyzable data format from the time stamp (t sa ) is subtracted from a point in time at the end of the recorded event. [5] Method according to claim 4, wherein the second processing time (D2) - is known or determined as a specific period of time, or - based on previous measurements of processing times and statistical methods, a probability is determined for a range of values ​​into which the second processing time (D2) falls. [6] Computer program which is designed to carry out individual or all steps of the method according to one of claims 1 to 5 by means of program code on a computing unit. [7] Computing unit on which the computer program according to claim 6 is implemented. [8] Computing unit according to claim 7, which is part of an in-vehicle computing unit of an at least partially automated vehicle, or which is part of an industrial processing plant.

Images