Method and system for a motor vehicle to verify the currency of an image of the vehicle's surroundings using a digital stamp.

Abstract

Method for verifying the currency of an image (1) of a vehicle environment (2) of a motor vehicle (3), comprising the steps: a) Generating the image (1) by means of an optical sensor unit (4) of the motor vehicle (3); b) Inserting a digital stamp (5) into the image (1) using the optical sensor unit (4), wherein the stamp (5) is inserted according to a time-dependent rule, wherein - the time-dependent rule specifies which pixel (15) of the image (1) is to be replaced by one or more pixels (15), - a color of the replacement pixel (15) or replacement pixels (15) is specified, - the incorporated stamp (5) of the image (1) is not recognizable to a user when the image (1) is displayed on a display unit (7), and - the stamp (5) used is small and / or has insufficient contrast for human perception; c) Transferring the image (1) with the stamp (5) from the optical sensor unit (4) to a receiving unit (6) of the motor vehicle (3); d) Checking the currency of the stamp (5) of the transmitted image (1) using the time-dependent rule by means of the receiving unit (6), wherein a time signal is generated by a central timer unit (8) of the motor vehicle (3) and is received by the optical sensor unit (4) and the receiving unit (6), wherein the time-dependent rule depends on the generated time signal.

Description

[0001] The invention relates to a method for verifying the recency of an image of a vehicle's surroundings. The invention further relates to an associated system comprising an optical sensor unit for generating the image and a receiver unit for receiving the image, as well as an associated motor vehicle.

[0002] Transmission errors can occur along a transmission path, for example, between an optical sensor unit of a vehicle and a receiver unit of the vehicle. These errors can be caused by disturbances in the transmission path. As a result, the receiver unit may receive incorrect, delayed, or frozen images. Consequently, it is possible that a driver, a driver assistance system, or an electronic vehicle control system could make a decision based on this incorrect information, potentially creating a dangerous situation.

[0003] DE10 2016 103 037 A1 relates to a method for checking the functionality of a camera-monitor system by means of additional information in the input image file.

[0004] DE 10 2016 202 620 A1 describes a method for monitoring a spatial area using a dynamic means.

[0005] DE 10 2014 008 808 A1 discloses a method for securing the transmission of security-relevant camera images, wherein a running counter is imprinted on the camera side.

[0006] US 2022 / 0217326 A1 concerns systems for monitoring the condition of imaging devices and procedures.

[0007] DE 10 2016 214 939 A1 discloses a method for controlling a safety-relevant function of a vehicle.

[0008] The invention is based on the objective of providing a method, a system and a vehicle by means of which it can be ensured that a received image is up-to-date.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are described by the dependent patent claims, the following description, and the figures.

[0010] One aspect of the invention provides a method for verifying the recency of an image of a vehicle's surroundings. The method specifically comprises the following steps: a) Generating the image using an optical sensor unit of the motor vehicle; b) Inserting a digital stamp into the image using the optical sensor unit, wherein the stamp is inserted according to a time-dependent rule; c) Transferring the image with the stamp from the optical sensor unit to a receiving unit of the vehicle; d) Checking the currency of the stamp of the transmitted image according to the time-dependent regulation using the receiving unit.

[0011] This method allows for extremely reliable and robust verification of the received image's accuracy. In other words, the image transmission path, from the optical sensor unit to the receiver, is end-to-end secured, ensuring that the received image can be assessed as current with absolute certainty. Consequently, the driver, driver assistance system, or electronic vehicle control system can make safe driving decisions based on the current image, which accurately reflects the vehicle's surroundings. This significantly enhances road safety for all road users.

[0012] For example, in a reversing parking situation, the optical sensor unit can be configured as or include a reversing camera, which captures and generates images of the area behind the vehicle. These images can then be displayed on a screen for the driver, allowing the driver to reverse the vehicle based on the displayed images. By checking the image's recency, the method ensures that the driver reverses the vehicle based on the current image.

[0013] The term "currentity" can also be understood as temporal originality or temporal validity. This verification can also be described as plausibility checks. In other words, the procedure is suitable for verifying the validity and temporal originality of the image.

[0014] The vehicle environment can be, in particular, the external environment of the motor vehicle. The optical sensor unit can, in particular, detect an area or section of the vehicle environment, for example, the area behind the vehicle, be a component of an advanced driver assistance system (ADAS) of the vehicle, and / or be mounted or integrated on the roof, front, or rear of the motor vehicle. The optical sensor unit can, for example, be or include a camera.

[0015] The optical sensor unit can be configured, in particular, to generate a single image of the vehicle's surroundings. Preferably, the optical sensor unit can be configured to generate a multitude of successive images, especially a video recording. Consequently, the method can be used to verify the recency of the multitude of successive images. In other words, the method can be used to verify whether the image or video recording is received by the receiving unit virtually in real time, i.e., "live." The generated image can, in particular, be a digital image and preferably comprise a multitude of pixels.

[0016] The digital stamp can be inserted into the image immediately after or simultaneously with its creation. In particular, the stamp can be inserted using the optical sensor unit that generates the image, ensuring that the stamp is already embedded in the image at the very beginning of the transmission.

[0017] The digital stamp can be permanently embedded in the image, meaning that once it has been embedded, the digital stamp no longer changes. Embedding the digital stamp in the image can be understood to mean, in particular, that at least one or more pixels of the image of the vehicle's surroundings are replaced by at least one or more pixels of the digital stamp.

[0018] In particular, the optical sensor unit, preferably a generation unit of the sensor unit, can generate the stamp according to the time-dependent rule and insert the generated stamp into the image. The time-dependent rule, which may also be referred to simply as the rule below, specifies which pixel(s) of the image are to be replaced by one or more pixels. The color of the pixel(s) according to the rule is also specified.

[0019] The regulation can be time-dependent, in particular. Preferably, the regulation can depend on the time at which the image was captured or the stamp is applied. In other words, the regulation can prescribe the application of the stamp as a function of time, especially how the stamp is to be applied to the image. This allows the application of the stamp to be at least indirectly dependent on time.

[0020] The image with the stamp can then be sent from the optical sensor unit and received by the receiver unit, for example, as a digital image signal. The image with the stamp can be transmitted from the optical sensor unit to the receiver unit via a transmission path. This transmission path can include, for example, a cable connection such as an Ethernet cable, routers, and / or electronic processing units that can, for example, convert or amplify the image signal to be transmitted.

[0021] The receiving unit at the end of the transmission path can be configured, in particular, to verify the recency of the received image, especially the recency of the image's stamp, whereby the verification can be carried out according to the known, time-dependent procedure. This verification allows an assessment of whether the image is current, i.e., transmitted in real time or live.

[0022] The receiving unit can be designed, for example, as a processing unit or as a module of a processing unit, such as an ADAS system in a motor vehicle. Alternatively or additionally, the receiving unit can be designed as a module of a display unit in the motor vehicle, such as a display in the vehicle's interior.

[0023] For example, the receiving unit can generate a stamp according to a time-dependent rule and compare the generated stamp with the stamp embedded in the image. The rule for generating the stamp can depend on the expected time of creation of the received image. With real-time image transmission, the expected time can correspond to the time of reception. Accordingly, an expected transmission time can be subtracted from the time of reception. In other words, the embedded stamp can be compared with an expected stamp to verify the currency of the stamp and thus the image. If, for example, the embedded stamp matches the expected stamp, the rule is fulfilled and the image can be considered current. If this is not the case, the image can be considered outdated.

[0024] For example, the receiving unit might be designed to recognize the inserted stamp in the image and, based on the time-dependent rule, reciprocally determine the time the image was created. This determined time can then be compared with the expected time of image creation to verify the stamp's, and therefore the image's, recency. If, for instance, the determined time matches the expected time, the rule is fulfilled and the image can be considered current. If not, the image can be considered outdated.

[0025] In particular, the validity of the stamp can also be verified in other ways, which are not explicitly mentioned here.

[0026] If, for example, the image is deemed outdated, a further step in the procedure could include not displaying the transmitted image on the display unit. Instead, an error message or similar could be displayed. Alternatively or additionally, the received image could not be processed further, for example, by the vehicle's ADAS system. For instance, a further step could involve initiating a fault analysis of the transmission link and / or reinitializing the transmission link.

[0027] The procedure can be carried out not only by motor vehicle systems, but also by other systems, for example by systems for object protection monitoring to check the recency of an image of an object's environment, or other camera-monitor systems.

[0028] One embodiment provides that the image is only displayed, for example by the display unit, if the stamp check shows that the stamp meets the time-dependent requirement. Another embodiment provides that the image is only processed further, for example by the ADAS system, if the stamp check shows that the stamp meets the time-dependent requirement.

[0029] One advantage of this implementation is that neither the driver nor the ADAS system makes incorrect decisions regarding vehicle control based on the faulty image transmission. Instead, only those images are displayed or processed whose recency has been reliably verified. This significantly increases road safety.

[0030] One embodiment provides that the embedded image stamp is not visible to a user, particularly a driver or passenger, when the image is displayed on a screen. This has the advantage, among others, that the user is not irritated or distracted by the stamp when viewing the image. Driving safety is therefore not impaired in any way.

[0031] In particular, the stamp is not visible when the user views the display unit as intended, for example, from an upright seated position. The stamp is not visible because it is small and / or lacks sufficient contrast for human perception, although the stamp can still be clearly detected by the receiving unit.

[0032] An alternative embodiment provides that the embedded image stamp is visible to the user when the image is displayed on a screen. This allows the user to verify that the image is current. This is particularly advantageous when dealing with a large number of consecutive images, as the stamp changes over time as the images are displayed, and the user can easily recognize these changes.

[0033] One embodiment provides that a graphic representation and / or the position of the stamp in the image is defined according to the time-dependent rule. Particularly with a large number of images, the graphic representation and / or the position of the stamp can thus change over time.

[0034] For example, the stamp can be graphically represented as a pointer on an imaginary clock, with the time-dependent rule defining the position of the pointer on the imaginary clock. For example, according to the rule, the pointer can be in a "12 o'clock" position at one point in time and in a "1 o'clock" position at a second point in time.

[0035] In particular, any graphic representation or position of the stamp is conceivable that can be defined according to the time-dependent regulation.

[0036] It is provided that a time signal is generated by a central timer unit of the vehicle and received by the optical sensor unit and the receiver unit. Preferably, the time-dependent instruction depends on the generated time signal.

[0037] In particular, the central timer unit includes a clock, and the time signal can contain information about the current time. The time signal can be generated in a clock cycle, for example, in seconds, tenths of a second, or milliseconds.

[0038] Preferably, the optical sensor unit and the receiver unit can be synchronized using the time signal. Specifically, the stamp is inserted according to the rule, which is dependent on the time signal. The stamp's recency is also checked against the rule, which is dependent on the time signal. This ensures that the transmitted image is transmitted in real time. In other words, the time signal can be used to verify that the image is not outdated or delayed. Driving decisions are therefore always advantageously based on a current image.

[0039] An alternative embodiment provides that a first image is generated at a first time point and a second image at a subsequent second time point using the optical sensor unit. Preferably, a first digital stamp is inserted into the first image and a second digital stamp into the second image using the sensor unit. Preferably, the first stamp is inserted according to the specification depending on the first time point, and the second stamp according to the specification depending on the second time point, so that the first and second stamps are configured differently. Preferably, the second stamp is checked for any difference or change compared to the first stamp.

[0040] For example, in the situation of a stationary vehicle, the camera can generate essentially identical images if the vehicle's surroundings do not change. Particularly when a large number of images are generated, a "still image" is produced in this situation. However, this embodiment can be used to verify that the image is still current and not frozen, since the stamp changes over time despite the static image. Thus, it can at least be ensured that the received image is not frozen.

[0041] One advantage of this embodiment is that a time signal is not required for synchronization, at least to determine whether the image is a "moving" image or a frozen image. Instead, the optical sensor unit and the receiver unit can independently apply or verify the stamp.

[0042] One embodiment involves the stamp being embedded with information about the time the image was created. For example, the stamp could be a code that can be read, at least machine-readable, for instance, by a reading unit in the receiving unit. It is also conceivable that the stamp could graphically represent a time, accurate to the second, which could also be read by a user. Advantageously, the time information can be directly compared with an expected time to verify the timeliness of the transmitted image.

[0043] Alternatively or additionally, the stamp can be incorporated with a pattern consisting of one or more pixels. If the pattern consists of only one pixel, for example, it can shift over time across multiple images, making changes to the stamp noticeable when comparing different pictures. This allows the stamp to be integrated into the image particularly easily and discreetly.

[0044] One embodiment involves placing the stamp in a peripheral area of ​​the image. An advantage of this is that the stamp leaves the essential image information untouched, so no image information is lost when the stamp is placed. Another advantage is that the stamp, being in the peripheral area, does not disturb or distract the user.

[0045] A further aspect of the invention provides a system for a motor vehicle to verify the recency of an image of the vehicle's surroundings. The system comprises at least one optical sensor unit for generating the image and one receiver unit for receiving the image. These can be connected to each other, in particular, by a transmission link for transmitting image signals and / or control signals. The optical sensor unit is specifically configured to insert a digital stamp into the image. The stamp is preferably programmable according to a time-dependent schedule. The receiver unit is specifically configured to verify the recency of the stamp in the received image based on the time-dependent schedule.

[0046] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0047] Further embodiments of the system according to the invention follow directly from the various configurations of the method according to the invention, and vice versa. In particular, individual features and corresponding explanations as well as advantages relating to the various configurations of the method according to the invention can be transferred analogously to corresponding configurations of the system according to the invention. In particular, the system according to the invention is configured or programmed to carry out a method according to the invention. In particular, the system according to the invention carries out the method according to the invention.

[0048] The system can also be used for an object, such as a building, to check the recency of an image of an object's environment, or for other applications.

[0049] A further aspect of the invention provides a motor vehicle which includes a system according to the invention.

[0050] The invention also includes combinations of the features of the described embodiments.

[0051] The following describes exemplary embodiments of the invention. This is illustrated by: Fig. 1 a flowchart of an embodiment of a method according to the invention; Fig. 2 a sketched representation of an embodiment of a method according to the invention; Fig. 3 a schematic representation of a stamp over time; Fig. 4 a schematic representation of an exemplary embodiment of a motor vehicle with a system.

[0052] The embodiments described below are preferred embodiments of the invention. In these embodiments, the described components each represent individual features of the invention that can be considered independently of one another. Each of these features further develops the invention independently and can therefore be considered part of the invention individually or in a combination other than that shown. Furthermore, the described embodiments can also be supplemented by other features of the invention already described.

[0053] In the figures, functionally identical elements are each provided with the same reference symbols.

[0054] Fig. Figure 1 shows a flowchart of an embodiment of a method according to the invention for verifying the recency of an image 1 of a vehicle environment 2 of a motor vehicle 3. In a first step S1, the image 1 is generated by means of an optical sensor unit 4 of the motor vehicle 3. In particular, the optical sensor unit 4 detects a section, for example a rear area 18 of the vehicle environment 2, and consequently generates the image 1 of the detected vehicle environment 2. For example, the optical sensor unit 4 is a reversing camera.

[0055] In a second step S2, a digital stamp 5 is inserted into the image 1 by means of the optical sensor unit 4, in particular by means of a module of the optical sensor unit 4. Preferably, the stamp 5 is inserted into the image 1 according to a time-dependent rule. For example, the stamp 5 is first generated by a generation unit of the optical sensor unit 4 according to the time-dependent rule and then inserted into the image 1. For this purpose, for example, one or more pixels of the image 1 of the vehicle environment 2 can be replaced by pixels of the stamp 5 in order to insert the stamp 5.

[0056] In a third step S3, the image 1 with the stamp 5 is transmitted from the optical sensor unit 4 to a receiver unit 6 of the vehicle 3, for example via a transmission link 19. The transmission link 19 can, for example, have wired and / or wireless transmission paths, as well as components 20 for processing and / or amplifying an image signal of the image 1. In this transmission link 19, or already in the optical sensor unit 4, disturbances can occur, for example, so that the image 1 may not be transmitted or may only be transmitted with a time delay.

[0057] To detect and counteract such disturbances, in a fourth step S4 the recency of the stamp 5 of the transmitted image 1 is checked against the time-dependent rule by the receiving unit 6. For example, the stamp 5 can be compared with an expected stamp 14 according to the time-dependent rule, which the receiving unit 6 has generated, by a reference unit of the receiving unit 6. For example, the image 1 can be considered current if the stamp 5 corresponds to the expected stamp 14.

[0058] For example, in a fifth step S5, image 1 can only be displayed and / or further processed on a display unit 7 if the result of the check of the stamp 5 shows that the stamp 5 meets the time-dependent requirement. If the stamp 5 does not meet the time-dependent requirement, an error message can be issued and / or a troubleshooting process initiated.

[0059] In the Fig. Figure 2 shows a sketched representation of an embodiment of a method according to the invention. For example, the optical sensor unit 4 generates the image 1 and places the stamp 5 in a border region 16 of the image 1. Subsequently, the image 1 with the stamp 5 can be transmitted to the receiving unit 6 via the transmission path 19, which can include one or more processing units 20. For example, the receiving unit 6 checks the validity of the stamp 5 based on the time-dependent rule, for example by generating the expected stamp 14 based on the time-dependent rule and comparing it with the transmitted stamp 5. If the test shows that the stamp 5 meets the rule, the image 1 can be displayed on the display unit 7.

[0060] Fig. Figure 3 shows a schematic representation of a stamp 5 over time, showing the stamp 5 being applied at each point in time along a time axis t. At a first time 11, a first stamp 5 can be applied according to the time-dependent rule. In this example, the stamp 5 comprises two black pixels 15, arranged one above the other with a gap of one pixel. The first stamp 5 can be applied to a first image 1, and preferably, once applied, the stamp 5 does not change.

[0061] At a second time point 12, which follows the first time point 11 by a time interval Δt, a second stamp 9 can be generated and applied according to the time-dependent rule. In this example, the second stamp 9 can comprise two adjacent pixels 15. At a third time point 13, which follows the second time point 12 by a time interval Δt, a third stamp 10 can be generated and applied according to the time-dependent rule. For example, a pattern of the stamp 5 can repeat itself over time after a multitude of time intervals Δt.

[0062] In the Fig.Figure 4 shows a schematic representation of an embodiment of a motor vehicle 3 with a system 17. The system 17 of the motor vehicle 3 can, in particular, comprise the optical sensor unit 4 and the receiver unit 6, which can be connected to each other via the transmission line 19. In this example, the optical sensor unit 4 comprises a reversing camera that can detect the area behind the vehicle 18 in the surroundings 2.

[0063] For example, the motor vehicle 3 can also include a central timing unit 8, which can be configured to generate a time signal that can be received by the optical sensor unit 4 and the receiver unit 6. Preferably, the time-dependent instruction, and thus the stamp, depends on the generated time signal. Reference symbol list 1 image 2 Vehicle environment 3 Motor vehicle 4 Optical Sensor Unit 5 Digital Stamp 6 receiver unit 7 Display unit 8 Timer unit 9 Second digital stamp 10 Third digital stamp 11 First point in time 12 Second point in time 13 Third point in time 14 Expected digital stamp 15 pixels 16 Edge area 17 System 18 Rear area 19 transmission line 20 processing units t time Δt time interval

Claims

Method for verifying the currency of an image (1) of a vehicle environment (2) of a motor vehicle (3), comprising the steps: a) generating the image (1) by means of an optical sensor unit (4) of the motor vehicle (3); b) introducing a digital stamp (5) into the image (1) by means of the optical sensor unit (4), wherein the stamp (5) is introduced according to a time-dependent rule, wherein: - the time-dependent rule specifies which pixel (15) of the image (1) is to be replaced by one or more pixels (15); - a color of the replacing pixel (15) or pixels (15) is specified; - the introduced stamp (5) of the image (1) is not perceptible to a user when the image (1) is displayed on a display unit (7); and - the introduced stamp (5) is small and / or has insufficient contrast for human perception;c) Transmitting the image (1) with the stamp (5) from the optical sensor unit (4) to a receiver unit (6) of the motor vehicle (3); d) Checking the recency of the stamp (5) of the transmitted image (1) according to the time-dependent rule by means of the receiver unit (6), wherein a time signal is generated by a central timer unit (8) of the motor vehicle (3) and received by the optical sensor unit (4) and the receiver unit (6), wherein the time-dependent rule depends on the generated time signal.; Method according to claim 1, characterized in that the image (1) is only displayed and / or further processed if a result of the inspection of the stamp (5) shows that the stamp (5) meets the time-dependent requirement. Method according to one of the preceding claims, characterized in that a graphic representation and / or a position of the stamp (5) is defined according to the time-dependent rule. Method according to one of the preceding claims, characterized in that the stamp (5) is introduced with information about a time of generation of the image (1), and / or the stamp (5) is introduced with a pattern consisting of one or more pixels (15). Method according to one of the preceding claims, characterized in that the stamp (5) is placed in an edge region (16) of the image (1). System (17) for a motor vehicle (3) for checking the recency of an image (1) of a vehicle environment (2), comprising an optical sensor unit (4) for generating the image (1) and a receiver unit (6) for receiving the image (1), characterized in that: - the optical sensor unit (4) is configured to introduce a digital stamp (5) into the generated image (1), wherein the stamp (5) can be introduced according to a time-dependent rule, wherein: - the time-dependent rule specifies which pixel (15) of the image (1) is to be replaced by one or more pixels (15); - a color of the replacing pixel (15) or pixels (15) is specified; - the introduced stamp (5) of the image (1) is not recognizable to a user when the image (1) is displayed on a display unit (7); - the introduced stamp (5) is small and / or has insufficient contrast for human perception;and- the receiving unit (6) is configured to check the recency of the stamp (5) of the received image (1) on the basis of the time-dependent rule, wherein a time signal is generated by a central timer unit (8) of the motor vehicle (3) and is received by the optical sensor unit (4) and the receiving unit (6), wherein the time-dependent rule depends on the generated time signal.; Motor vehicle (3) comprising a system (17) according to claim 6.

Images