Image repetition detection technology

Abstract

A method for image repetition detection, comprising: Receiving (S101) a video signal having temporally successive individual images (10, 20), each individual image comprising a plurality of pixels (34); Dividing (S102) each of the individual images into sub-images (32, 38); Calculating (S103) a first intensity value for a first partial image (32) of a first individual image (10) based on a true subset of the pixels (34) of the first partial image (10) and calculating (S104) a second intensity value for a second partial image (38) of a second individual image (20) based on the true subset of the pixels (34) of the second partial image (38), - wherein a position of the second partial image (38) within the second individual image (20) corresponds to a position of the first partial image (32) within the first individual image (10) and - wherein a position of the pixels (34) of the true subset within the second partial image (38) corresponds to a position of the pixels (34) of the true subset within the first partial image (10); Comparing (S105) the first intensity value with the second intensity value; and Detecting (S106) an image repetition based on the comparison, wherein at least some of the pixels (34) of the true subset are arranged along at least one straight line (36) within the respective partial image (32, 38), and wherein the at least one straight line (36) is a diagonal within the respective partial image (32, 38).

Description

Technical area

[0001] The present disclosure generally relates to a method for image repetition detection, for example, for detecting the repetition of images from a vehicle camera. Furthermore, the present disclosure relates to a device for image repetition detection, a system comprising the device, and a motor vehicle comprising the device. background

[0002] In modern vehicles, rearview and surround-view cameras are part of safety-relevant assistance systems. To ensure the safety of the vehicle's occupants and those around them, it is essential that these cameras function correctly. In any case, a fault must be reliably detected. This problem also occurs in other technical components in which cameras are installed.

[0003] The output signal from a camera, which is of interest to a vehicle control unit for autonomous driving, for example, is processed serially in several steps, for example, by encoding. It is important that each processing component that processes the output signal or a signal derived from it can rely on the functionality and integrity of the preceding processing component.

[0004] A complete camera failure can be easily detected using various mechanisms. However, due to, for example, a software error in a processing component downstream of the camera, it may happen that the same images are repeatedly output. As a result, the processing components downstream of this faulty processing component can no longer detect any changes in the image data, even though the scene captured by the camera may well have changed. Such a failure can lead to serious accidents or other problems, especially in autonomous vehicles, but also in other technical areas.

[0005] Further prior art is known from the documents DE 10 2011 117 654 A1 and DE 10 2004 001 719 A1. The first of these documents (DE 10 2011 117 654 A1) describes a method for operating an image processing device, in which an image signal superimposed with a noise signal is transmitted between a transmitting device and at least one receiving device, and the image signal comprises temporally successive images, each with an image content and a noise pattern of the noise signal superimposed on the image content. Provision is made for this to occur: after transmission, at least two consecutive images of the image signal are compared with each other, at least in regions, and if a difference is found between the noise patterns contained in the images, a successful image transmission is detected, even if the images have identical image content.

[0006] From the publication DE 10 2004 001 719 A1, a method for checking video camera systems generating digital image data for the occurrence of image freeze is known. The method comprises the following steps: a) analyzing the temporal progression of bit values ​​of a predetermined selection of pixel positions in a frame sequence for fluctuations during operation of the video camera system, and b) detecting image freeze if a value determined from the fluctuations of the bit values ​​of the selected, non-overdriven pixel positions is lower than a predetermined value derived from the noise at the pixel positions of the image sensor.

[0007] Document CN 108 416 759 A discloses a method for detecting display errors. The display errors to be detected relate exclusively to errors that can occur due to a defect in a display unit. These errors are detected by comparing the color values ​​of predetermined pixels of an individual image before and after its display by the display unit. An error is detected if the displayed color value does not correspond to the value to be displayed. Brief outline

[0008] The present disclosure is based on the object of carrying out efficient image repetition detection, in particular in real time.

[0009] According to a first aspect, a method for image repetition detection is provided. The method comprises receiving a video signal with temporally successive individual images, wherein each individual image comprises a plurality of pixels, and dividing each of the individual images into sub-images. For a first sub-image of a first individual image, a first intensity value is calculated based on a true subset of the pixels of the first sub-image. For a second sub-image of a second individual image, a second intensity value is calculated based on the true subset of the pixels of the second sub-image. A position of the second sub-image within the second individual image corresponds to a position of the first sub-image within the first individual image, and a position of the pixels of the true subset within the second individual image corresponds to a position of the pixels of the true subset within the first sub-image.The method further comprises comparing the first intensity value with the second intensity value and detecting an image repetition based on the comparison.

[0010] The video signal can be received directly by a camera. Alternatively, the video signal can be digitally preprocessed (e.g., encoded) by a component downstream of the camera and received accordingly in a preprocessed format. In this case, the image data output by the camera can be combined into individual data packets by the downstream component, for example (e.g., video encoding can result in such data packets).

[0011] If the video signal is received directly from a video camera, the video signal can be directly divided into a number of temporally consecutive individual images. In the case of digital preprocessing, such as encoding (e.g., for compression), decoding (e.g., decompression) is required to divide the data packets into temporally consecutive individual images.

[0012] Each of the individual images consists of a plurality of digital image elements, called pixels. The pixels can be arranged in a grid. Each of these pixels is assigned at least one value that represents the intensity (e.g., brightness or color) of the pixel perceived by a viewer. The overall impression of the respective individual image results from the sum of the pixel intensities perceived by a viewer.

[0013] Each of the individual frames obtained from the video signal is divided into sub-frames in the subsequent process step. According to one variant, the number of sub-frames is the same for each individual frame of the video signal. Thus, if the first received frame is divided into a number of k sub-frames, each of the subsequent frames can also be divided into k sub-frames.

[0014] As mentioned above, each of the individual images consists of a plurality of pixels. Accordingly, each of the sub-images also consists of a plurality of pixels. From this plurality of pixels of a particular sub-image, a true subset of pixels is selected that is smaller than (i.e., not equal to) the total number of pixels in the first sub-image. The number of pixels in the true subset is the same for each sub-image of each of the individual images. Based on the values ​​representative of the intensities of the pixels in the true subset, the first intensity value is calculated for the first sub-image, and similarly for the second sub-image.

[0015] At least some of the pixels of the true subset are arranged along at least one straight line within the respective partial image. In the case of a rectangular (e.g. square) partial image, these can be, for example, those pixels of the partial image that are arranged along an image edge of the partial image. In the case of a triangular partial image, these can be, for example, those pixels of the partial image that are arranged along a side bisector or angle bisector of the triangle. In the case of partial images of any shape, these can be, for example, those pixels of the partial image that are arranged along a straight line that connects two opposite sides of the partial image. In particular, at least some of the pixels of the true subset are arranged along at least one diagonal within the respective partial image.

[0016] According to a second aspect, a method for image repetition detection is provided analogous to the first aspect. However, in this case, the individual images are divided into square sub-images. All image edges of the sub-images are of equal length and arranged at right angles to each other. Along the sides of the square sub-images, N = 2 n pixels, where N is a natural number. The square sub-images in this case consist of 2 2n pixels.

[0017] For example, 2 or 4 or 8 or 16 or 32 or 64 pixels can be arranged along the sides of the square sub-images.

[0018] Each of the pixels can be assigned at least one pixel value that influences its display. The first and second intensity values ​​can be calculated from those pixel values ​​that are assigned to the true subset of pixels of the respective partial image. In some embodiments, the at least one pixel value is selected from a red value (R), a green value (G), a blue value (B), a grayscale value, and an alpha value (α). The R, G, and B values ​​influence the color of the respective pixel as perceived by a viewer. The grayscale value influences the brightness of the respective pixel as perceived by a viewer. The alpha value influences the transparency of the respective pixel as perceived by a viewer.

[0019] Often, a pixel is assigned a tuple of the values ​​mentioned or other values, such as (R, G, B) or (R, G, B, a). In this case, the respective intensity value can be calculated from some or all of the values ​​of the tuple. For example, in the examples mentioned above, the values ​​for R, G, and B can be summed for each pixel to calculate the intensity value assigned to that pixel. The intensity value assigned to a sub-image can in turn be calculated by summing the intensity values ​​determined for the pixels of the corresponding true subset. If at least one of the pixel values ​​described above is assigned to the pixels of the true subset of the respective sub-image, the calculation of the first and second intensity values ​​therefore comprises, for example, summing the respective pixel values ​​of the pixels of the true subset.

[0020] In some embodiments, the first and second intensity values ​​can each be an integer value. In some embodiments, it can be recognized that an image repetition does not occur if the comparison of the first intensity value with the second intensity value does not result in a match. In some embodiments, the comparison of the respective first intensity value with the respective second intensity value can be carried out for two or more partial images of the first individual image and the second individual image. The respective partial images can be arranged logically consecutively within the respective individual image. The partial images available for comparison can, as a whole, result in the respective individual image. Alternatively, the respective partial images can be arranged at individual, predetermined positions within the respective individual image.There may therefore be gaps in the image between these positions that should not be used for comparison.

[0021] It can be recognized that an image repetition is present if the comparison of the respective first intensity value with the respective second intensity value for all sub-images of the first individual image and the second individual image results in at least an approximate match. If an exact match (especially of integer) intensity values ​​is required, it can be recognized particularly reliably that an image repetition is present, which is due to technical problems, e.g., an encoder. If the system functions correctly, the image noise that is unavoidable in static scenes already leads to different intensity values ​​of consecutive individual images, which can rule out an image repetition.

[0022] In the above embodiment, the first intensity value of various first partial images of the first individual image can be compared with the second intensity value of second partial images of the second individual image, each corresponding to the second intensity value, until a specific comparison does not result in at least an approximate (in particular, an exact) match. If an at least approximate (in particular, an exact) match of the intensity values ​​of all partial images considered is found, an image repetition can be detected.

[0023] In a first variant of the method, following the determined comparison, no further intensity values ​​are calculated for partial images of the first and second individual images for which no comparison has yet been performed. Thus, as soon as the presence of no image repetition has been positively detected, the method can be aborted according to this variant. The next subsequent (third) individual image can then be processed immediately to subject it to the comparison, thus avoiding unnecessary delays in the process.

[0024] In an alternative second variant of the method, following the determined comparison, further intensity values ​​are calculated and stored for additional partial images of at least the second individual image for which no comparison has yet been performed. In this alternative variant, intensity values ​​are calculated for at least some additional partial images of the second individual image, even though it has already been positively determined that there is no image repetition.

[0025] The alternative second variant is useful, for example, if a memory field for the most recently calculated (e.g., first) intensity value for the most recently viewed (e.g., first) individual image is defined in a memory area for all viewed sub-image positions of an individual image. This stored intensity value can then be read out in order to compare it with the corresponding freshly calculated (e.g., second) intensity value of the just received (e.g., second) individual image. Following the comparison, the most recently calculated intensity value is overwritten with the freshly calculated intensity value. Normally, this overwriting only occurs with regard to the intensity values ​​for the newly received individual image during one process run until at least an approximate match is not found.However, the old intensity values ​​of the last viewed frame remain in the remaining memory fields. This fact can, in some cases, slow down image repetition detection to the extent that a large number of identical frames must be compared before an image repetition can actually be detected. To avoid or at least mitigate this disadvantage, in the second variant, intensity values ​​are calculated and stored for at least some (especially for all) additional frames of the second frame, even though it has already been positively detected that there is no image repetition.

[0026] In the second variant of the method, the second frame can follow the first frame in time. One or more of the intensity values ​​stored for the subsequent partial frames can then be used to detect image repetition between the second frame and a third frame that follows the second frame in time.

[0027] The process execution according to the first variant and the second variant can alternate cyclically or non-cyclically. In particular, a certain number of, for example, 2 qMethod executions according to the first variant are followed by method executions according to the second variant, where q is a natural number. For example, the method can first be performed for four partial images of the first and second individual images according to the second embodiment; thus, termination of the method execution upon detection of an image repetition can be avoided for the first four partial images. The method execution according to the first embodiment then takes place.

[0028] According to a third aspect, a device for image repetition detection, in particular in a vehicle camera, is presented. The device comprises an input interface designed to receive a video signal with temporally successive individual images, wherein each individual image comprises a plurality of pixels. The device further comprises an evaluation device designed to divide each of the individual images into partial images and to calculate a first intensity value for a first partial image of a first individual image based on a true subset of the pixels of the first partial image and to calculate a second intensity value for a second partial image of a second individual image based on the true subset of the pixels of the second partial image.A position of the second partial image within the second individual image corresponds to a position of the first partial image within the first individual image, and a position of the pixels of the true subset within the second partial image corresponds to a position of the pixels of the true subset within the first partial image. The evaluation device is designed to compare the first intensity value with the second intensity value and to detect an image repetition based on the comparison. At least some of the pixels of the true subset are arranged along at least one straight line within the respective partial image. The at least one straight line is a diagonal within the respective partial image.

[0029] According to a fourth aspect, a device for image repetition detection is provided analogous to the third aspect. Here, the individual images are divided into square partial images, and for a number N of pixels arranged along the sides of the square partial images, the following applies: N = 2 n , where n is a natural number.

[0030] The device according to the third and / or fourth aspect may further be configured to output the video signal to a vehicle control unit if no image repetition has been detected and / or output an error message to the / a vehicle control unit if image repetition has been detected.

[0031] The input interface can be designed as a hardware interface, in particular as a plug-in contact. Alternatively or additionally, the device can comprise an output interface designed as a hardware interface, in particular as a plug-in contact.

[0032] According to a fifth aspect, a system is presented that comprises a device for image repetition detection according to the third and / or fourth aspect, a camera for generating image data, and an encoding device for generating an encoded video signal from the image data. In one variant, the input interface of the device for image repetition detection according to the second aspect is connected downstream of the encoding device.

[0033] According to a sixth aspect, a motor vehicle is presented which comprises a device for image repetition detection according to the third and / or fourth aspect, a vehicle camera coupled to the input interface of the device and a vehicle control unit coupled to an output interface of the device. Short description of the drawings

[0034] Further advantages, details, and features of the present disclosure will become apparent from the following description of exemplary embodiments and from the figures. They show: Fig. 1A shows a representation of a first frame of a video signal with different image areas having different intensities; Fig. 1B is a representation of a second frame of the video signal which differs from the frame according to the representation of Fig. 1A differs; Fig. 2 shows a representation of a first square partial image of the first individual image, wherein a proper subset of pixels of the first square partial image is formed from the pixels arranged along a diagonal of the square partial image; Fig. 3 is a flowchart of an embodiment of a method for image repetition detection; Fig. 4A a more detailed flow diagram of the Fig. 3 procedure; Fig. 4B shows a first part of the flowchart of the Fig. 4A; Fig. 4C shows a second part of the flow chart of the Fig. 4A; and Fig. 5 shows an embodiment of a module for image repetition detection. Detailed description

[0035] It is well known that every camera that captures digital images produces image noise. Even in static scenes, the values ​​of at least some pixels in successive individual images differ from one another. The formation of image noise and the optical effects underlying it are well known to those skilled in the art and require no further explanation. The pixels that generate image noise are evenly distributed throughout the entire image captured by the camera and are often invisible to the naked eye.

[0036] The primary objective of the method and device according to the following embodiments is not to detect image noise as such. Rather, the reproduction of two completely identical individual images is to be detected, for example, due to the failure of an image processing component connected downstream of the camera (which is often integrated with the camera to form a camera module). This problem can occur, for example, if an encoding device connected downstream of the camera for generating an encoded video signal from image data received from the camera malfunctions, such as a software defect. In this case, often only the last individual image transmitted before the malfunction occurs is continuously output.The downstream systems therefore assume that there is still image data available for processing, even though this image data no longer allows any conclusions to be drawn about the scene actually captured by the camera. It is obvious that such an error could lead to accidents, for example, in autonomous and semi-autonomous vehicles.

[0037] To detect such errors, one can take advantage of the uniform distribution of pixels subject to image noise described above. Due to the uniform distribution of pixels subject to image noise, each sufficiently large sample of consecutive individual images contains a pixel subject to image noise. If the number and position of the pixels in the sample taken from each individual image are identical and the samples do not differ from each other, it can be assumed that the images are completely identical, which indicates an error. Otherwise, the compared samples would differ due to image noise.

[0038] However, the method described above for detecting image repetition requires, in the worst case, comparing consecutive individual images pixel by pixel. Due to the high computational complexity, this approach is impractical, especially for real-time applications such as those essential for autonomous and semi-autonomous vehicle systems.

[0039] Therefore, a method and a module for image repetition detection are described below using various exemplary embodiments that solve the problem explained above. The method is applied, for example, to the video signal output by a vehicle camera module. The image repetition detection module is connected, for example, between the vehicle camera module and a vehicle control unit (e.g., via plug contacts).

[0040] If the various embodiments have the same or similar components, these will be referred to below with the same reference numerals for the sake of simplicity.

[0041] The Fig. Figure 1A shows the representation of a first individual image 10 consisting of pixels, which was recorded, for example, by a vehicle camera. The first individual image 10 has different image areas 12, 14, 16, and 18, which are perceived by a viewer with different intensities. The different intensities are shown in the Fig. 1A with different hatching of the individual image areas 12, 14, 16, 18, 20 and can refer to colors, brightnesses, etc. of the pixels of the individual image 10.

[0042] The Fig. Figure 1B shows the representation of a second frame 20 that immediately follows the first frame 10. The second frame 20 also has different image areas 22, 24, 26, and 28, which are perceived by a viewer with different intensities. Analogous to the representation of Fig. 1A are the different intensities in the Fig. 1B with different hatching of the individual image areas 22, 24, 26 and 28. As can be seen from a comparison of the Fig. 1A and Fig. 1B, the image areas 12, 14, 16 and 18 differ slightly from the image areas 22, 24, 26 and 28. The first individual image 10 and the second individual image 20 are therefore different, so there is no image repetition caused by the malfunction of a component downstream of the camera described above.

[0043] However, if the image areas 12, 14, 16, and 18 are completely identical to the image areas 22, 24, 26, and 28 with regard to their respective pixel values, it can be assumed that the image repetition is caused by a system error. To determine this image repetition beyond doubt, the method discussed below proposes first dividing each of the individual images 10, 20 into a plurality of partial images, as described in the Fig. 1A and Fig. 1B, the merely indicated rasterization 30 of the individual images 10, 20, which corresponds in position, is illustrated in partial images 32, 38.

[0044] In the Fig. 2 shows a first partial image 32 of the first individual image 10. The first partial image 32 corresponds to the upper left partial image of the Fig. 1A indicated grid 30 and is in the example according to Fig. 2 a square partial image. For example, the first partial image 32 can correspond exactly to a square of the Fig. 1A and Fig. 1B. However, other shapes of partial images, such as rectangular or triangular partial images, are also encompassed by the present disclosure.

[0045] As in the Fig. 2, an area of ​​the first partial image 32 is also provided with the hatching of the image area 18, thus having the same intensity perceived by a viewer as the image area 18. Along the sides of the square first partial image 32, 2 4= 16 pixels 34. The first partial image 32 thus consists of a total of 16 by 16 pixels 34. However, according to the present disclosure, it is also possible for a different number of pixels 34 to be arranged along the sides of the square partial images. Preferably, 2 pixels 34 are arranged along the sides of the square partial images. n The pixels are arranged in 34 pixels, where n is a natural number. This choice of sub-image size simplifies the computational operations and thus favors real-time processing.

[0046] Of further interest in the present embodiment are in particular the pixels 34 arranged along the two diagonals 36 of the square partial image 32, which in the representation of the Fig. 2 are highlighted merely for illustrative purposes. These pixels 34 arranged along the diagonal 36 of the square partial image 32 form a true subset of the total number of pixels 34 arranged within the first partial image 32. In other embodiments not shown, the pixels 34 forming the true subset can, for example, also be arranged along one or two side edges of the square partial image 32 or along two straight lines, which in turn subdivide the square partial image into four smaller squares.

[0047] As already described above, each of the pixels 34 of the partial images is assigned a pixel value that influences its display. For example, each of the pixels 34 can be provided with a gray value (e.g., between 0 and 255). Each pixel can also be assigned a tuple of an R, G, and B value (e.g., each between 0 and 255). According to the present exemplary embodiment, not every pixel value of the pixels 34 arranged within the first partial image 32 is taken into account, but rather only the pixel values ​​of the pixels 34 arranged along the diagonal 36. From the totality of these pixel values, a first intensity value for the first partial image 32 is calculated, for example, by summing up the individual pixel values ​​in a manner that requires little computational effort.

[0048] For a second partial image 38 of the second individual image 20 (not shown) which also consists of 16 by 16 pixels 34, a second intensity value is calculated in the same way from the pixel values ​​of the pixels 34 arranged along the diagonal 36. The position of the second partial image 38 within the second individual image 20 corresponds to the position of the first partial image 32 within the first individual image 10, as can be seen from an examination of the Fig. 1B. The partial images 32, 38 are therefore in the same position. A comparison of the first and second intensity values ​​now makes it possible to detect an image repetition with respect to the first and second individual images 10, 20.

[0049] Since only the pixels 34 arranged along the diagonal 36 are included in the calculation of the first and second intensity values, rather than all of the 16 x 16 pixels 34 of each partial image, the computational effort required to detect an image repetition is significantly reduced. Furthermore, it can be assumed that the pixels 34 arranged along the diagonal 36 represent most of the different pixel values ​​occurring within the partial image 32. Restricting the calculation to the pixels 34 arranged along the diagonal 36 thus offers, on the one hand, an efficient comparison option for different partial images in terms of computational effort, while still taking into account most of the different pixel values ​​occurring within the partial images.

[0050] The individual process steps of a method for image repetition detection based on the above explanations are shown in the flow chart according to the Fig. 3. The method can be carried out, for example, by a processor, in particular a central processing unit (CPU) or a graphical processing unit (GPU).

[0051] In step S101, at least two temporally consecutive individual images 10, 20 are received. These individual images 10, 20 were described above with reference to the Fig. 1A and Fig. 1B explains.

[0052] The individual images 10, 20 are divided in step S102 into at least two partial images 32, 38. The partial images 32, 38 can, for example, be divided according to the Fig. 2 be square and a number of 2 n times 2 n pixels 34.

[0053] In steps S103 and S104, according to the above explanations, Fig. 2, a first and second intensity value are calculated. The first and second intensity values ​​are compared in step S105. In step S106, an image repetition is detected based on the comparison. Specifically, for example, if a deviation between the two intensity values ​​is detected, it is determined that no image repetition is present.

[0054] In the Fig. 4A, Fig. 4B and Fig. 4C, the individual steps of the procedure are described in Fig. 3 explained in more detail. The Fig. 4B shows a method implementation in the case that in step S106 according to the method implementation according to Fig. 3 an at least approximate match between the first and second intensity values ​​was detected. Fig. 4C, however, shows a method implementation in the case that in step S106 according to the method implementation according to Fig. 3 no at least approximate agreement between the first and second intensity values ​​was detected.

[0055] The Fig. The method illustrated in Figure 4A begins with receiving individual images in step S201 and dividing the individual images into sub-images in step S202. Steps S201 and S202 correspond to steps S101 and S102 of the method implementation according to Fig. 3.

[0056] It should be noted that in some implementations, in step S201, only two consecutive individual images 10, 20 are received during the very first iteration. During the second, third, etc., iteration, only a single additional individual image needs to be received and divided into sub-images for intensity value enrichment, since the sub-image intensity values ​​for the immediately preceding individual image 20 have already been calculated (and stored) in the previous iteration.

[0057] In step S203, the initial value of a variable "STATE" is set to "UNCHANGED." The variable "STATE" indicates the presence or absence of an image repetition.

[0058] After setting the starting value of the variable ZUSTAT, the procedure reaches the entry point A. Depending on the further course of the procedure, it is possible to jump back to this entry point A during the procedure execution.

[0059] After the entry point A, the pairwise intensity value comparison of corresponding partial images of different individual images follows. The comparison of the respective partial images begins again with the calculation of the first intensity value for a first partial image 32 of the first individual image 10 (step S204) and the calculation of the second intensity value for a second partial image 38 of the second individual image 20 (step S205), as described above in connection with the Fig. 2 and Fig. 3. The respective positions of the partial images 32, 38 in the individual images 10, 20 are identical, as are the positions of the pixels used to calculate the intensity values.

[0060] In the iterative implementation of the intensity value comparison presented here, the intensity value for the first partial image 32 may already have been calculated in a previous iteration (i.e., in a comparison for a single image preceding the first individual image 10 with the first individual image 10). In this case, the corresponding intensity value for the first partial image 32 can be stored in a memory immediately after its calculation in the previous iteration and loaded from the memory again in step S204.

[0061] The two intensity values ​​calculated or loaded in steps S204 and S205 are compared with each other in step S206. In step S207, a check is made as to whether the comparison of the first and second intensity values ​​according to step S206 resulted in a match between the first and second intensity values. If this is not the case (i.e., there is no image repetition), the method is repeated according to the method implementation of Fig. 4C. See transition point C in the Fig. 4A and Fig. 4C.

[0062] If, however, in step S207 an at least approximate and (in the case of integer intensity values, in particular an exact) match between the first and the second intensity value is determined, i.e. an image repetition is possibly present, the method is carried out according to the method implementation of the Fig. 4B. See transition point B in the Fig. 4A and Fig. 4B.

[0063] In the following, the procedure is first described in accordance with Fig. 4C, i.e., when there is no image repetition. The procedure according to Fig. 4C begins with step S208. In step S208, the value of the variable STATE is set to "CHANGED." In other words, the process flow signals that the absence of an image repetition has been detected.

[0064] The second intensity value for which at least an approximate match with a first intensity value was determined in step S207 is stored in the memory mentioned above in step S209 as a reference value for further comparisons with intensity values ​​of partial images of the same position in a third individual image following the second individual image 20 in time, so that it does not have to be recalculated during the next iteration. For this purpose, an intensity value memory field can be defined in a memory area of ​​the memory for all viewable partial image positions of an individual image. The intensity values ​​of possibly previous comparisons do not need to be stored again, since these comparisons already yielded a match (see step S207).In other words, the intensity values ​​assigned to a specific sub-image position in the memory area have not changed, so that the memory field contents are still up to date.

[0065] In step S210, a check is made to determine whether premature termination of the process is permitted. "Premature" means that an intensity value is not calculated for all partial images of the last received frame 20 (which is not necessary if, for example, a match between the individual images can already be denied when comparing the logically first partial images of both frames 10, 20). A premature termination of the process can be prevented, for example, every m frame comparisons, with, for example, m = 2 q , where q is a natural number.

[0066] If, however, a premature termination of the process is permitted, the process execution for the first and second individual images 10, 20 can be terminated. The system then waits for the next individual image and restarts with step S201. Otherwise, the process execution is continued according to the Fig. 4B, so that a current intensity value can be calculated for all partial images of the currently viewed individual image 20 in step S209 and stored in the memory area in the correct position. See the transition point B in the Fig. 4B and Fig. 4C. The review of whether early termination of proceedings is permitted is discussed in more detail below.

[0067] The procedure according to Fig. 4B begins with step S211. In step S211, a check is made to determine whether further partial images 32, 38 of the first and second individual images 10, 20 are present. If the check in step S211 yields a positive result, the method execution according to steps S204 to S210 is repeated for the logically next partial images of the individual images 10, 20. The method execution is thus continued at entry point A. If the jump to transition point B occurs starting from the method variant according to Fig. 4C, further intensity values ​​for further partial images of the second individual image 20 are then calculated and stored as a reference value for comparison with third partial images of a subsequent third individual image. Thus, a forced update of the intensity values ​​of all partial images of the second individual image 20 takes place, even though image repetition could already be ruled out in the preceding step S207.

[0068] Preventing premature termination of the update of all intensity values ​​for the current frame 20 despite a detected difference from the previous frame 10 increases the time required to execute the current cycle, which is actually undesirable, especially in real-time applications. On the other hand, this approach allows for a free choice of an upper time limit for the error state detection of an image repetition depending on the parameters m or q explained above.

[0069] Without the forced update of all partial image intensity values ​​for the current frame 20, the number of frames required for error condition detection is, in the worst case, equal to the number of partial images in a frame. This is precisely the case when the last abort has already occurred for the first partial image considered and, apart from the corresponding currently calculated and saved (see step 209) intensity value, all other partial image intensity values ​​previously stored in memory therefore differ from the intensity values ​​of the current frame 20.Due to the deviation of all stored subsequent partial image intensity values, no image repetition is then successively detected in the next individual image for the second partial image under consideration (but another intensity value is correctly updated and saved), in the subsequent individual image only for the third partial image under consideration, and so on, until all intensity values ​​could be correctly updated.

[0070] In practice, depending on the frame rate of the camera used and the number of partial images per frame, the process described above can require several seconds of continuous reception of identical individual images before an image repetition can actually be detected. This time is too long for many applications. By appropriately selecting the parameter m or q, however, a forced update can take place every 500 ms (or shorter), for example, by not allowing premature termination (step S210). In this way, the maximum duration of error condition detection is also limited to 500 ms (or shorter). As the parameter mk or q increases, the computational effort decreases, while the number of iterations for error condition detection increases.

[0071] If, however, the check in step S211 produces a negative result, i.e., there are no further partial images of the second individual image, the value of the variable ZUSTAT is checked in step S212. If the value of the variable ZUSTAT was not changed during the entire execution of the method (i.e., the value of the variable ZUSTAT is still UNCHANGED), this indicates that at least two consecutive individual images have been repeated. In this case, an error message is output in step S213, for example, to a vehicle control unit. If, however, the value of the variable ZUSTAT was changed during the execution of the method, this indicates that the consecutive individual images have not been repeated. In this case, the current iteration can be ended, and the next iteration can begin with the receipt of the next individual image.

[0072] Fig. Figure 5 shows an embodiment of a device 300 for image repetition detection. The device 300 comprises an input interface 302, an evaluation device 304, and an output interface 310. The interfaces 304, 310 can be designed as plug contacts. The device 300 is designed to carry out the Fig. 2 to 4 described image repetition detection procedure.

[0073] The input interface 302 is configured to receive a video signal with temporally successive individual images from a camera 312, for example, a vehicle camera. The camera 312 can be embodied as a camera module, which can include a device for encoding the image data recorded by the actual camera 312 according to a video coding standard. The corresponding camera module thus outputs an encoded video signal.

[0074] The evaluation device 304 comprises a repetition detection module 306 and a logical image gate 308. The received video signal is forwarded by the input interface 302 to both the repetition detection module 306 and the logical image gate 308. If the repetition detection module 306 detects a repetition of an image according to the method described above, it outputs an error status (e.g., by setting a flag) to a vehicle control unit 314 logically connected downstream of the device 300 via a logical channel 310a. The vehicle control unit 314 can initiate appropriate countermeasures based on the error signal to counteract the negative effects of an erroneous image repetition. Furthermore, if an image repetition is detected, the repetition detection module 306 sends an error signal (e.g., by setting a flag) to the logical image gate 308.The error signal causes the image gate 308 to close. In other words, when an image repetition is detected, the video signal transmitted by the camera 312 is not forwarded by the device 300 to the vehicle control unit 314.

[0075] If, however, the image repetition detection module 306 detects that there is no image repetition, no signal (and no error condition) is output to the vehicle control unit 314 via the logical channel 310a. In this case, the image repetition detection module 306 transmits a confirmation signal to the image gate 308. Based on the confirmation signal, the image gate 308 forwards the video signal received from the camera 312 unchanged to the vehicle control unit 314 via a logical channel 310b of the output interface 310.

[0076] The input interface 302 and / or the output interface 310 can be configured as a hardware interface, in particular as a plug-in contact. This allows the device 300 to be connected to a plurality of cameras 312 and / or vehicle control units 314 via the respective hardware interface. Accordingly, the device 300 can be used independently of a camera 312 and / or a vehicle control unit 314. In particular, this enables retrofitting of cameras 312 and / or control units 314 already installed in a vehicle with the device 300.

[0077] According to the embodiments presented here, intensity values ​​are calculated only for a true subset of all pixels of a partial image. According to one embodiment, the pixels of the true subset are arranged along the diagonals of a square partial image. This reduces the memory requirements (by up to a factor of 2). 2n) and the computational effort required for image repetition detection is also reduced (by up to a factor of 2 n-1 ). At the same time, the pixels on the sub-image diagonal can be determined by a simple bit shift corresponding to a given number of pixels, and thus implemented with little computational effort.

[0078] Furthermore, the individual images received from a camera are divided into a number 2 n divided into sub-images, and for a number of 2 q During the execution of the procedure, a forced update of the intensity values ​​of the partial images of the last received frame takes place. Due to the powers of two used, all operations required for the procedure can be represented as bit shifts or AND / OR operators. This type of representation of operations is particularly efficient for processors to calculate.

[0079] Finally, by appropriately selecting the parameters n and m or q, the process can be adapted to different camera systems and / or vehicle control units. For example, increasing the parameter m reduces the memory and computational effort, but increases the probability of false detection of the repeated image. On the other hand, increasing the parameter q reduces the computational effort, while increasing the number of iterations required for repeated image detection.

[0080] The solution presented here is particularly suitable for real-time applications, for example for motor vehicles or robotics.

Claims

[1] A method for image repetition detection, comprising: Receiving (S101) a video signal having temporally successive individual images (10, 20), each individual image comprising a plurality of pixels (34); Dividing (S102) each of the individual images into sub-images (32, 38); Calculating (S103) a first intensity value for a first partial image (32) of a first individual image (10) based on a true subset of the pixels (34) of the first partial image (10) and calculating (S104) a second intensity value for a second partial image (38) of a second individual image (20) based on the true subset of the pixels (34) of the second partial image (38), - wherein a position of the second partial image (38) within the second individual image (20) corresponds to a position of the first partial image (32) within the first individual image (10) and - wherein a position of the pixels (34) of the true subset within the second partial image (38) corresponds to a position of the pixels (34) of the true subset within the first partial image (10); Comparing (S105) the first intensity value with the second intensity value; and Detecting (S106) an image repetition based on the comparison, wherein at least some of the pixels (34) of the true subset are arranged along at least one straight line (36) within the respective partial image (32, 38), and wherein the at least one straight line (36) is a diagonal within the respective partial image (32, 38). [2] A method for image repetition detection, comprising: Receiving (S101) a video signal having temporally successive individual images (10, 20), each individual image comprising a plurality of pixels (34); Dividing (S102) each of the individual images into sub-images (32, 38); Calculating (S103) a first intensity value for a first partial image (32) of a first individual image (10) based on a true subset of the pixels (34) of the first partial image (10) and calculating (S104) a second intensity value for a second partial image (38) of a second individual image (20) based on the true subset of the pixels (34) of the second partial image (38), - wherein a position of the second partial image (38) within the second individual image (20) corresponds to a position of the first partial image (32) within the first individual image (10) and - wherein a position of the pixels (34) of the true subset within the second partial image (38) corresponds to a position of the pixels (34) of the true subset within the first partial image (10); Comparing (S105) the first intensity value with the second intensity value; and Detecting (S106) an image repetition based on the comparison, wherein the division (S102) of the individual images into square partial images (32, 38) takes place, and wherein for a number N of pixels (34) arranged along sides of the square partial images (32, 38), the following applies: N=2n, where n is a natural number. [3] Method according to one of the preceding claims, wherein each of the pixels (34) is assigned at least one pixel value which influences its display; wherein the first and second intensity values ​​are calculated from the pixel values ​​associated with the true subset of pixels (34) of the respective partial image (32, 38). [4] The method of claim 3, wherein the at least one pixel value is selected from the following set of values: red value, green value, blue value, grayscale value and alpha value. [5] Method according to claim 3 or 4, wherein the calculation of the first and second intensity values ​​comprises summing the pixel values ​​associated with the true subset of pixels (34) of the respective partial image (32, 38). [6] Method according to one of the preceding claims, wherein the first and second intensity values ​​are each an integer value. [7] Method according to one of the preceding claims, wherein it is recognized that an image repetition does not occur if the comparison (S105) of the first intensity value with the second intensity value does not result in a match. [8] Method according to one of the preceding claims, wherein the comparison (S105) of the respective first intensity value with the respective second intensity value is carried out for two or more partial images (32, 38) of the first individual image (10) and the second individual image (20). [9] Method according to claim 8, wherein it is recognized that an image repetition is present if the comparison (S105) of the respective first intensity value with the respective second intensity value for all partial images of the first individual image (10) and the second individual image (20) results in at least an approximate match. [10] Method according to claims 8 and 9, wherein the first intensity value of different first partial images of the first individual image (10) is compared with second partial images of the second individual image (20) corresponding to the second intensity value, until no at least approximate agreement is obtained in a specific comparison. [11] Method according to claim 10, wherein following the determined comparison, no further intensity values ​​are calculated for partial images (32, 38) of the first and second individual images (10, 20) for which no comparison has yet taken place. [12] Method according to claim 10, wherein following the determined comparison, further intensity values ​​are calculated and stored for further partial images of at least the second individual image (20) for which no comparison has yet taken place. [13] The method of claim 12, wherein the second individual image (20) follows the first individual image (10) in time, wherein one or more of the intensity values ​​stored for the further partial images are used for image repetition detection between the second individual image (20) and a third individual image following the second individual image in time. [14] Method according to claims 11 and 12, wherein the process implementation according to claim 11 and the process implementation according to claim 12 alternate, in particular a number 2 qof method implementations according to claim 12, a method implementation according to claim 11 follows, where q is a natural number. [15] Device (300) for image repetition detection, comprising: an input interface (302) configured to receive a video signal comprising temporally successive individual images (10, 20), each individual image (10, 20) comprising a plurality of pixels (34); and an evaluation device (304, 306, 308) which is designed to: - dividing each of the individual images into sub-images (32, 38); - Calculating a first intensity value for a first partial image (32) of a first individual image (10) based on a true subset of the pixels (34) of the first partial image (32) and calculating a second intensity value for a second partial image (38) of a second individual image (20) based on the true subset of the pixels (34) of the second partial image (38), wherein a position of the second partial image (38) within the second individual image (20) corresponds to a position of the first partial image (32) within the first individual image (10) and wherein a position of the pixels (34) of the true subset within the second partial image (38) corresponds to a position of the pixels (34) of the true subset within the first partial image (32); - comparing the first intensity value with the second intensity value; and - detecting an image repetition based on the comparison, wherein at least some of the pixels (34) of the true subset are arranged along at least one straight line (36) within the respective partial image (32, 38), and wherein the at least one straight line (36) is a diagonal within the respective partial image (32, 38). [16] Device (300) for image repetition detection, comprising: an input interface (302) configured to receive a video signal comprising temporally successive individual images (10, 20), each individual image (10, 20) comprising a plurality of pixels (34); and an evaluation device (304, 306, 308) which is designed to: - dividing each of the individual images into sub-images (32, 38); - Calculating a first intensity value for a first partial image (32) of a first individual image (10) based on a true subset of the pixels (34) of the first partial image (32) and calculating a second intensity value for a second partial image (38) of a second individual image (20) based on the true subset of the pixels (34) of the second partial image (38), wherein a position of the second partial image (38) within the second individual image (20) corresponds to a position of the first partial image (32) within the first individual image (10) and wherein a position of the pixels (34) of the true subset within the second partial image (38) corresponds to a position of the pixels (34) of the true subset within the first partial image (32); - comparing the first intensity value with the second intensity value; and - Detecting an image repetition based on the comparison, where the individual images are divided into square partial images (32, 38), and wherein for a number N of pixels (34) arranged along sides of the square partial images (32, 38), the following applies: N=2n, where n is a natural number. [17] Device (300) according to one of claims 15 or 16, designed for Outputting the video signal to a vehicle control unit (314) if no image repetition was detected and / or Outputting an error message to the / a vehicle control unit (314) if an image repetition is detected. [18] Device (300) according to one of claims 15 to 17, wherein the input interface (302) is designed as a hardware interface, in particular as a plug-in contact, and / or wherein the device (300) comprises an output interface (310) which is designed as a hardware interface, in particular as a plug contact. [19] A system comprising a device (300) for image repetition detection according to any one of claims 15 to 18, a camera (312) for generating image data and an encoding device for generating an encoded video signal from the image data, wherein the input interface (302) of the device for image repetition detection is connected downstream of the encoding device. [20] Motor vehicle, comprising a device (300) for image repetition detection according to one of claims 15 to 18, a vehicle camera (312) coupled to the input interface (302) of the device (300) and a vehicle control unit (314) coupled to an output interface (310) of the device.

Images