Removing burn-in from full-motion video images for video applications

Abstract

A system and method for removing burn-in from full motion video (FMV) imagery. In some cases, the technique is a pre-processing step in forensic or military applications. The system and method identifies one or more burn-in overlay areas in a full motion video image and creates a mask of the one or more burn-in overlay areas. A matched intensity is created for a number of pixels in the mask. In some cases, to create the matched intensity, inpainting is used in a central portion of the full motion video image and ordinal filtering is used on the periphery of the full motion video image.

Description

[Technical field]

[0001] Government Interest Statement

[0001] This disclosure was made with U.S. Government support under Contract No. W56KGU-15-D-0007 under Technical Information Engineering Services (TIES) Subcontract No. S18-10080 awarded by the U.S. Army Communications-Electronics Research Development and Engineering Center (CERDEC). The U.S. Government has certain rights in this invention.

[0002]

[0002] The present disclosure relates to full-motion video (FMV) image processing, and more particularly to burn-in removal techniques for full-motion video imagery for use in further video exploitation pipelines useful for threat detection, automatic target recognition, and the like. [Background technology]

[0003]

[0003] In almost all motion full motion videos (FMV), there is burn-in that incorporates metadata into the imagery. In some examples, sensor metadata such as location coordinates (latitude / longitude / altitude), platform yaw / pitch / roll, and other user interface / geospatial data are burned as an overlay onto the FMV imagery as part of the video collection process. In some embodiments, the burn-in may include any or a subset of the following overlays onto the collected imagery: (i) collection metadata such as date, timestamp, frame number, location, (ii) sensor metadata such as modality, field of view (FOV), line of sight (LOS), (iii) user interface (UI) data such as crosshairs, focus area, mode of operation (e.g., auto / manual), (iv) geospatial data such as scale, north marker and / or arrow, (v) readings from inertial navigation systems (INS), telemetry / radar systems, gyrometers, accelerometers, system error indicators, etc.

[0004]

[0004] Automated target detection, tracking and recognition algorithms are required to process these videos with burned-in sensor metadata and other UI / geospatial data due to the clutter introduced by these overlays. The presence of burn-in adversely affects frame-to-frame registration during automated video exploitation of FMV imagery. Frame-to-frame registration aids moving target detection, the main basis for many tracking algorithms. Frame-to-frame registration to correct for apparent camera motion (jitter) extracts feature points from the current frame and corresponding feature points in nearby image areas in the previous frame. Matching these feature points in successive frames aligns the frames with each other. Burn-in tends to overlay a large amount of text and numbers on the image, whose image edges / corners induce a large number of spurious feature points. This causes frames to be aligned primarily based on feature points from the burn-in and not from real image features. Burn-in therefore impairs alignment of successive frames, i.e., inter-frame homography. Burn-in is therefore an obstacle to frame-to-frame registration and needs to be removed to accurately align frames for further downstream use for target detection, tracking, etc. in an automated exploitation pipeline.

[0005]

[0005] The presence of burn-in also has a negative impact on background modeling in video exploitation systems. Accurate background modeling is necessary to identify foreground areas of an image to perform detection and tracking in an automated video exploitation pipeline. The background is generally learned or modeled from the image, and frames are differentiald to perform background subtraction to identify moving areas of the image as motion-blobs. Blob detection refers to detecting areas in a digital image that have different properties, such as brightness or color, compared to the surrounding areas. Some of the burn-in is often dynamic. For example, north markers and arrows move around the image as the camera moves during collection. Instrument readings displayed as burn-in, location coordinates, etc. are constantly changing in full-motion video. This interferes with background modeling, and furthermore, when frame differencing is performed to subtract the background, these changes can cause spurious motion blobs that result in a large number of false alarms and errors. Burn-in is therefore an obstacle to background modeling / subtraction and therefore needs to be removed to accurately perform target detection, tracking, etc. in an automated video exploitation pipeline.

[0006]

[0006] In a further example, metadata symbols are often burned into aerial imagery that should be reviewed by analysts. Burn-ins from classified data generally need to be redacted. Current systems simply black out these areas by superimposing a black box on the burn-ins. However, automated target detection, tracking and recognition algorithms find the resulting redacted image to be messy and prone to false positives and similar errors due to the artificial edges and corners introduced by the black box. This obstacle is currently addressed by preventing the overlay areas from being processed, instead of physically removing the overlay. Therefore, it is generally not feasible to utilize this existing solution with video exploitation systems to properly process the entire image content.

[0007] It is therefore an object of the present disclosure to overcome the above-mentioned shortcomings and drawbacks associated with conventional full-motion video image processing. Summary of the Invention

[0008]

[0008] One aspect of the present disclosure is a method for removal of burn-in data from a full-motion video image, the method comprising: identifying one or more burn-in overlay areas in a full-motion video image; creating a mask of the one or more burn-in overlay areas; dilating the mask, the mask comprising a plurality of pixels; providing at least a portion of the dilated mask to a pixel replacement section; creating adapted intensities for a plurality of pixels in the portion of the dilated mask based on a plurality of neighboring pixels, the portion comprising the one or more burn-in overlay areas and a plurality of neighboring pixels proximate to the one or more burn-in overlay areas; and replacing the one or more burn-in overlay areas by filling a plurality of pixels in the one or more burn-in overlay areas with the adapted intensities to form a resultant image.

[0009]

[0009] One embodiment of the method further comprises subjecting the obtained images to video processing to obtain one or more detections.

[0010]

[0010] Another embodiment of the method is one in which the acquired images are prepared for a video processing step in real time.

[0011]

[0011] In another embodiment of the method, the mask is created using a color space decomposition method. In some cases, the mask generated via color space decomposition begins by converting the RGB image to the YCbCr color space. In some embodiments, a binary mask is created. In some embodiments, the binary mask is fed to an inpainting algorithm.

[0012]

[0012] In yet another embodiment of the method, the automatic replacement of one or more burned areas with matched pixels is via inpainting and order-filtering. In some cases, inpainting is used in the center of the full-motion video image and order-filtering is used on the periphery of the full-motion video image.

[0013]

[0013] Another aspect of the disclosure is a computer program product including one or more non-transitory machine-readable media having instructions encoded thereon that, when executed by one or more processors, cause a process for removal of burn-in from a full motion video image to be performed, the process comprising identifying one or more burn-in overlay areas in the full motion video image; creating a mask of the one or more burn-in overlay areas; expanding the mask, the mask comprising a plurality of pixels; feeding at least a portion of the mask to an inpainting algorithm; creating adapted intensities for the plurality of pixels in the mask based on neighboring unmasked pixels; removing the one or more burn-in overlay areas by filling the plurality of pixels in the mask with the adapted intensities to form a resultant image; and subjecting the resultant image to video processing to obtain one or more detections.

[0014]

[0014] An embodiment of the computer program product is one in which the obtained images are prepared for a video processing step in real time.

[0015]

[0015] Another embodiment of the computer program product is one in which the mask is created using a color space decomposition method. In some cases, the mask generated via color space decomposition begins by converting the RGB image to a YCbCr color space. In some cases, a binary mask is created. In some embodiments, the binary mask is fed to an inpainting algorithm.

[0016]

[0016] In yet another embodiment of the computer program product, the automatic replacement of one or more burned areas with adapted pixels is via inpainting and sequential filtering. In some embodiments, inpainting is used in a central portion of the full-motion video image and sequential filtering is used on the periphery of the full-motion video image.

[0017]

[0017] Yet another aspect of the present disclosure is a method for removal of burn-in from a full motion video image, the method comprising: identifying one or more burn-in overlay areas in a full motion video image; creating a mask of the one or more burn-in overlay areas; expanding the mask, the mask comprising a plurality of pixels; feeding at least a portion of the mask to an inpainting algorithm; creating adapted intensities for the plurality of pixels in the mask based on neighboring unmasked pixels; removing the one or more burn-in overlay areas by filling the plurality of pixels in the mask with the adapted intensities to form a resultant image, wherein inpainting is used in a central portion of the full motion video image and ordinal filtering is used on the periphery of the full motion video image to create the adapted intensities; and subjecting the resultant image to a video processing step to obtain one or more detections.

[0018]

[0018] These aspects of the present disclosure are not intended to be exclusive, and other features, aspects and advantages of the present disclosure will be readily apparent to those of ordinary skill in the art when read in conjunction with the following description, the appended claims, and the accompanying drawings.

[0019]

[0019] The above and other objects, features, and advantages of the present disclosure will be apparent from the following description of specific embodiments of the disclosure as illustrated in the accompanying drawings, in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed on illustrating the principles of the present disclosure. The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee. [Brief description of the drawings]

[0020] [Figure 1]

[0020] FIG. 1 illustrates an image of a full-motion video before burn-in removal, according to one embodiment of the present disclosure. [Figure 2A]

[0021] FIG. 2 illustrates a color space mask applied to the image of the full-motion video in FIG. 1 before burn-in removal, according to one embodiment of the present disclosure. [Figure 2B] FIG. 2 illustrates a color space mask applied to the image of the full-motion video in FIG. 1 before burn-in removal, according to one embodiment of the present disclosure. [Diagram 3]

[0022] FIG. 4 illustrates a binarized mask of the full-motion video image from FIG. 2A or FIG. 2B before burn-in removal, according to one embodiment of the present disclosure. [Figure 4A]

[0023] FIG. 4 illustrates a morphological process applied to the image from FIG. 3 to expand a mask for burn-in removal from a full-motion video image, according to one embodiment of the present disclosure. [Figure 4B] FIG. 4 illustrates a morphological process applied to the image from FIG. 3 to expand a mask for burn-in removal from a full-motion video image, according to one embodiment of the present disclosure. [Diagram 5]

[0024] FIG. 4C illustrates a geodesic distance map of the full motion video image from FIG. 4B before burn-in removal, where neighboring pixels provide image information to the masked areas, according to one embodiment of the present disclosure. [Figure 6]

[0025] FIG. 13 illustrates an image of a full-motion video after burn-in removal, according to one embodiment of the present disclosure. [Figure 7]

[0026] 1 is a flow chart of a method for removing burn-in from full-motion video imagery as a pre-processing step, according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021]

[0027] In contrast to existing solutions, an embodiment of the present disclosure replaces burn-in in a full motion video (FMV) image with matched image information from neighboring pixels, thus erasing the burn-in overlay portion of the image with matched information that is consistent with the surrounding image area. Replacement of burn-in pixels that is consistent with the surrounding image area is important because it prevents false edges or corners from being formed, thus preventing the possibility that false positives may occur during automated feature extraction by the video exploitation algorithm. The resulting image (with the burn-in removed) can then be applied for further automated processing (the resulting image is a pre-processing step) or for viewing by an analyst for manual exploitation / processing. In contrast to previous methods, the resulting image described herein is useful in its entirety, since it appears to be seamlessly filled.

[0022]

[0028] Previous systems used a black box to "edit" the burn-in information. As used herein, a better alternative is to inpaint the burned area with image information from neighboring pixels. This makes the resulting image more visually appealing and provides additional benefits to downstream processing of the resulting image, such as through automated detection and tracking algorithms.

[0023]

[0029] An embodiment of a system and method for burn-in removal from FMV images for video exploitation automatically identifies and replaces burn-in areas in FMV images. Furthermore, the system can be used as a pre-processing step in an automated video processing pipeline. An embodiment of a method for burn-in removal from FMV images for video processing involves the following steps: (i) automatically identifying burn-in areas to create a mask via a color space decomposition method; (ii) automatically replacing the burn-in areas by filling masked pixels with intensity matched to their neighboring unmasked areas via image inpainting and sequential filtering; and (iii) incorporating this process as a pre-processing step into an existing video exploitation / processing pipeline.

[0024]

[0030] It should be appreciated that the techniques of the present disclosure are useful in forensic situations and / or in scenarios where add-on capabilities such as automatic detection, tracking and target recognition are required in addition to manual viewing. The techniques may also be used to create data test sets from data with burn-in. Some embodiments of the present disclosure are useful for defense industry and / or law enforcement uses where removal of metadata information burned into video image data is required so that analysts may extract intelligence products and / or exploit information in the imagery via automated systems for localization, tracking, scene analysis, machine learning, knowledge gathering purposes, etc. Other embodiments of the present disclosure may be useful for general commercial use where all or part of the present disclosure may be useful for removal of text overlays from video clips, such as subtitles, logos, and watermarks.

[0025]

[0031] One embodiment of the present disclosure for removing burn-in from full-motion video imagery involves (i) identifying burn-in overlay areas to create a mask, (ii) removing the overlay by filling the masked pixels with an intensity matched to their neighboring unmasked areas, and (iii) then subjecting the image to video processing to obtain successful detections. In some cases, the detections are of vehicles, creatures, or other areas / items of interest.

[0026]

[0032] In one embodiment, one step for the removal of burn-in from full-motion video imagery is to identify areas of the image with burn-in overlay and create a binary mask of these areas, so that the binary mask can be used to determine which areas need to be filled, such as by an inpainting algorithm. In one embodiment, the mask is generated via a color space decomposition method, where the first step is to convert the RGB image to the YCbCr color space. In the YCbCr format, luminance information is stored as a single component (Y) and chrominance information is stored as two color difference components (Cb and Cr). The Cb channel represents the difference between the blue component and a reference value. The Cr channel represents the difference between the red component and a reference value. Assuming a digital pixel represented in RGB format, with 8 bits per sample, where 0 and 255 represent black and white, respectively, the YCbCr components can be obtained according to equations (1)-(3).

[0027]

number

[0028]

[0033] Referring to FIG. 1, a full-motion video image 10 is shown prior to burn-in removal according to one embodiment of the present disclosure. More specifically, the image is a given image with selected burn-in data 2 shown as green. This burn-in data in one example represents sensitive data that needs to be removed but without compromising the underlying image. Burn-in data can also be information about the image, such as data provided to an operator, that adversely affects computer vision processing for automated identification and tracking. The white dotted boxes indicate areas of burn-in overlay, e.g., 4, 6, 8, 12, 14, 16, 18. As mentioned above, the boxes can vary in size depending on the extent of the burn-in data and are meant to include the burn-in content in a given area.

[0029]

[0034] Still referring to FIG. 1, the size of the boxes in one example is simplified, and larger boxes are used to capture isolated burn-in data within the boxes, such as that shown in region box 16. In another example, the boxes may be defined with smaller boxes (6, 8) capturing smaller portions of the burn-in data. Although shown as squares and rectangles, the boxes may be other shapes, so long as the burn-in content is captured within the region. In one embodiment, the identification of the burn-in overlay area involves mask formation, including thresholding the Cr component at a predetermined value for binarization.

[0030]

[0035] Referring to FIG. 2A, a full-motion video image 20 is shown with a mask of the overlay areas and prior to burn-in removal according to one embodiment of the present disclosure. More specifically, the Cr chrominance channel of the original image (from FIG. 1) is shown converted from RGB to YCbCr color space, with the burn-in overlay areas in boxes 4', 6', 8', 12', 14', 16', 18' standing out in cyan against a yellow background. The Cr component of the YCbCr color space highlights the burn-in areas in contrast to the remaining actual image areas. Thus, appropriate thresholding of the Cr channel of the YCbCr image allows a binary mask of the burn-in areas to be formed (see, for example, FIG. 3) to distinguish the burn-in areas from the underlying image areas.

[0031]

[0036] The threshold is determined by the chrominance range of the burned-in area highlighted in the Cr component image. In FIG. 2A, on the right side, a color bar is plotted adjacent to the image of the Cr channel of the YCbCr color space image. In this embodiment, the color bar indicates that chrominance values ​​of about 120 and below belong to the burned-in area, and higher values ​​belong to the actual image area. It is possible to use other color spaces for finding the mask, such as the LAB color space (shown in FIG. 2B).

[0032]

[0037] Referring to FIG. 2B, a full-motion video image 20' with a mask and before burn-in removal according to one embodiment of the present disclosure is shown. More specifically, the A channel of the original image (from FIG. 1) is converted from RGB to LAB color space and shown with the burn-in overlay areas highlighted in cyan against a yellow background and within boxes 4", 6", 8", 12", 14", 16", 18". Here, the A channel of the LAB image was thresholded at -5 to obtain the mask, as shown by the color bar in FIG. 2B. The above technique is an embodiment of the color separation process. The threshold value can be varied in different cases depending on the RGB color intensity of the burn-in present in the image. It is also possible to automate the threshold selection.

[0033]

[0038] Referring to Figure 3, a binarized mask 30 for a full-motion image before burn-in removal is shown according to one embodiment of the present disclosure. More specifically, here burn-in areas previously identified by thresholding (see, for example, Figure 2A or Figure 2B) are then selected to be highlighted on the image in boxes 21-27. The burn-in mask areas are highlighted in white, showing the final mask of burn-in areas as a binary image for subsequent processing steps.

[0034]

[0039] With reference to Figures 4A and 4B, a morphological process is shown applied to dilate a mask for burn-in removal from a full-motion video image, according to one embodiment of the present disclosure. More specifically, a binary mask 40 is shown in Figure 4A prior to the dilation / blurring / smoothing step shown in 40' of Figure 4B. The binary mask image is similar to the binary mask image of Figure 3. In some cases, the dilation / blurring / smoothing applied to the image is in the form of dilation. The dilation uses a structuring element to add pixels to the mask area boundary. The size and shape of the structuring element can vary. In some embodiments, a square structuring element of dimension 5 was used. The recommended structuring element size for color separation masking is 3, 5 or 7, with the square shape better suited to the nature of the burn-in area due to the presence of text and numbers. In some embodiments, a cumulative mean mask and two dilation steps in GIMP (GNU Image Manipulation Program) resulted in the mask shown in Figure 4B. Looking at the dotted box area 42 in FIG. 4A and then the same box area 42″ in FIG. 4B, it is clear that there has been some spreading / expansion of the burning data within the masked area. This expansion step helps ensure that the area to be masked, i.e., where the text or other burning ends and the image begins, is considered and complete.

[0035]

[0040] Referring to FIG. 5, a geodesic distance map 50 of a masked full-motion video image before burn-in removal is shown, with neighboring pixels providing image information to the masked area, according to one embodiment of the present disclosure. The process described herein is used to generate an underlying video image obscured by the burn-in data, so that the underlying image can be used in an automated detection and tracking process. Pixels shown as darker areas immediately surrounding the white burn-in mask area are weighted proportionally more heavily than the lighter red and yellow areas further away, calculating the weights that contribute to the fill intensity. Here, the white burn-in mask area is shown within a dotted box (e.g., 51-57) for clarity of labeling and illustration.

[0036]

[0041] In some embodiments, the inpainting strength of pixel p is determined by the values ​​of known pixels close to p. Its contribution is proportional to the distance of the neighboring pixels, determined by adding the estimates of all pixels q in the neighborhood, weighted by a normalized weighting function. The weighting function takes into account the image gradient as well as the distance from p. In some cases, this is an inpainting technique (e.g., Telea inpainting) that moves over the pixel's neighborhood based on an iterative algorithm, e.g., FMM (Fast Marching Method). Inpainting techniques are generally capable of handling color images, but for purposes herein, the images were converted from RGB to grayscale prior to inpainting, since video exploitation systems generally utilize grayscale images.

[0037]

[0042] In some embodiments, the processing time for the color space decomposition process to create the mask is one-third of the time it takes for other methods, such as image decomposition using FFT (Fast Fourier Transform). In one embodiment, the average time was about 0.04 seconds for the former versus about 0.13 seconds for the latter. Note that in order for existing video exploitation systems to be able to successfully use this preprocessing step, the preprocessing must be completed within some frame rate constraints of the system. Thus, faster methods employing color space decomposition as detailed herein are preferred over slower methods.

[0038]

[0043] To remove the burned-in overlay areas, according to one embodiment of the present disclosure, the masked pixels (i.e., the white areas within the white dotted box in FIG. 5) are filled with an intensity adapted to the neighboring image patches, thus replacing the burned-in overlay pixel intensities. In some cases, two different techniques are used for this. In one embodiment, the mask areas corresponding to the central portion 55 of the image (which generally contains the target or area of ​​interest) are precision-filled using an inpainting technique, and the peripheral portions of the mask (e.g., 53) are fast-filled using an ordinal filtering technique. The reason for using two different techniques is to ensure that the filling operation occurs quickly enough to facilitate real-time processing of the full-motion video image for further video processing.

[0039]

[0044] Inpainting results in better image quality, but it is a time-consuming operation. That is why in some embodiments, inpainting is restricted to the central portion of the frame. Ordinal filtering, on the other hand, results in more coarsely filled areas, but it is a much faster operation. Having a coarser fill in the background areas where there are no moving targets and a finer fill in the areas where the target of interest will be works well for real-time processing. In one embodiment, the average time taken for inpainting was about 2 seconds to about 2.5 seconds for an entire frame. When inpainting is restricted to the center of the image (e.g., the central 1 / 4 to 3 / 4 for a 1280x720 image), it takes less than 0.6 seconds.

[0040]

[0045] In one embodiment, the Telea inpainting algorithm was used. The Telea algorithm estimates the value of a pixel to be inpainted based on a neighborhood of known pixels around it. In some embodiments, an FMM algorithm was then used to determine the order in which the border pixels are inpainted, i.e., from smallest to largest distance to a known pixel, so as to reduce the size of the inpainting area as the algorithm progresses. In one embodiment, the algorithm starts at the boundary of the known neighborhood area to fill from the boundary first, and gradually moves inside the mask area. This takes a small neighborhood around the pixel to be inpainted. This pixel is replaced by a normalized weighted sum of all known pixels in the neighborhood. In some cases, the algorithm gives more weight to neighboring pixels that are close to the boundary normal, close to the point to be inpainted, and those that are on the boundary contour. Still referring to FIG. 5, the geodesic distance map of the masked original image shows the pixels as darker areas immediately around the white burned mask area, which are weighted proportionally heavier in contributing to the inpainting fill strength than lighter areas (such as red) further away.

[0041]

[0046] Referring to Fig. 6, a full-motion video image 60 after burn-in removal is shown according to one embodiment of the present disclosure. More specifically, Fig. 6 shows the result of applying a process including an inpainting algorithm on a given original image shown in Fig. 1, where the inpainting process replaced all overlay pixels with adapted intensities. This results in an image that appears complete, but no longer contains burn-in overlay areas that would be an obstacle for downstream processing. See, for example, boxes 64, 66, 68, 70, 72, 74, and 76, where the image has been approximated and is free of burn-in data.

[0042]

[0047] Some embodiments of burn-in removal from full-motion video imagery for video exploitation used an ordinal filtering algorithm for a fast filling process with which the area around the mask was filled. The algorithm was provided with an "ord" parameter and a "domain" parameter indicating the neighborhood to be filled. Ordinal filtering replaces each element in a given neighborhood by the "ord"-th element in a sorted set of neighbors specified by a non-zero element in the domain. The operation ordfilt2(A, 5, ones (3,3)) uses a 3x3 domain and orders the nine pixel intensity values ​​in the neighborhood in ascending order. It then replaces every pixel in the domain by the fifth element in the order. This example is the same as median filtering, since the fifth element is the median value. One embodiment of the ordinal filter allows for the selection of different elements in the order, not necessarily the median.

[0043]

[0048] Still referring to FIG. 6, the result of a combination of ordinal filtering and inpainting algorithms applied to the image shown in FIG. 1 is shown, where the inpainting process replaces all central overlay pixels (e.g., those in the mask in 72) and the peripheral areas (e.g., those in the mask in 64, 68...) are ordinal filtered. Here, 100 ordinal and 15x15 domains were used. The ordinal filtering operation takes about 0.24 seconds on average, making the total processing time for each frame less than 1 second. Since this technique is often used as a pre-processing step, it was verified that video processing is indeed possible to detect targets on the resulting inpainted and ordinal filtered images.

[0044]

[0049] The average time taken per frame to form the mask was about 0.04 seconds. The total average time taken for the inpainting, sequential filtering, and combination operations was about 0.83 seconds. Thus, the total processing time per frame was less than 1 second per frame in one embodiment. In general, when this type of process is ported to C++, there is a speedup of the order of 10, which makes the processing time per frame less than 1 / 10 of a second. If the video processing system processes video at about 6 Hz, this order of timing should be possible for real-time operation.

[0045]

[0050] The C++ implementation ran at 30-38 frames per second on 1024x768 resolution images. The implementation utilized the open source image processing library OpenCV. Although it was easy to port from MATLAB to C++, the algorithm required small modifications for C++ and for use in the video processing chain. No suitable ordinal filter function was available, so instead, a median filter was used to fast fill the image periphery. However, the median filter produced black areas in solid parts of the mask, where the masked area was larger than the kernel size. To solve this problem, the median filter applied the filter to both the mask and the image, and then ignored the pixels in the mask when calculating the neighborhood average by dividing the filtered image by the filtered mask. Furthermore, to avoid erroneously inpainting large parts of the image, the C++ implementation skipped inpainting if the percentage of pixels detected as mask exceeded a parameterized threshold.

[0046]

[0051] This disclosure shows that (a) color space decomposition methods can successfully identify burned-in overlay, (b) masked overlay areas can be successfully filled in less than one second per frame using a combination of inpainting and sequential filtering methods, and (c) video processing can successfully detect on the inpainted image with sufficient quality for the detection to then be used for track stitching.

[0047]

[0052] Referring to FIG. 7, a flow chart of a method for removing burn-in from full-motion video according to one embodiment of the present disclosure is shown. More specifically, the system identifies 100 one or more burn-in overlay areas in a full-motion video image. The burn-in overlay areas represent regions having burn-in data disposed on the image. The regions may be a number of pixels that incorporate one or more burn-in data elements and form a box or other shape, such as an ellipse or circle. The size of the area may vary depending on certain parameters and the space between the respective burn-in data items. The process creates a mask 102 of one or more burn-in overlay areas in the image. The mask comprises a number of pixels for the one or more burn-in areas. The pixels in the burn-in overlay areas contain burn-in data that occupy overlay area pixels that may have originally had image data but were later overwritten by the burn-in data. In some embodiments, the mask is generated via a color space decomposition method. In some cases, by converting the RGB image to a YCbCr color space. In some cases, the mask is then converted to a binary mask. In some embodiments, the binary mask is dilated 104 or, in some cases, blurred or expanded. Dilation refers to adding pixels to the mask area boundary using a structuring element. The size and shape of the structuring element can vary.

[0048]

[0053] Still referring to FIG. 7, at least a portion of the dilated mask is then provided to a pixel replacement section 106. The pixel replacement section utilizes a sequential filtering process, an inpainting process, or a combination of sequential filtering for some pixels and inpainting for other pixels. For pixels representing a static background, sequential filtering tends to be faster, while for pixels representing moving objects, inpainting processes tend to produce better quality. A dilated mask, or adapted intensities for pixels in the overlay area replacing the burned data, is created using neighboring pixels 108. The dilated mask refers to the burned overlay region as well as some neighboring pixels that may not have burned data but are in close proximity to the burned data, and thus are selected in the mask such that the burned data is not left unprocessed. The adapted intensities are selected as shown in FIG. 5. One or more burned overlay areas are replaced 110, such as by filling pixels in the dilated mask with adapted intensity pixels, to form a resultant image. The resulting image represents the starting video image, with the burn-in data replaced by an estimate of the original image based on a pixel replacement process. In some embodiments, the resulting image may then be subjected to a video processing step to obtain one or more detections. Detection in one example refers to target detection, such as buildings or vehicles, in the image. For example, if there is a fleet of vehicles, a tracking detection can isolate a particular vehicle through the moving video.

[0049]

[0054] Various inventive concepts may be implemented as one or more methods, examples of which are provided. The acts performed as part of a method may be ordered in any suitable manner. Thus, although acts are shown as sequential in an example embodiment, embodiments may be constructed in which acts are performed in an order different from that illustrated, which may include performing some acts simultaneously.

[0050]

[0055] Although various inventive embodiments have been described and illustrated herein, those skilled in the art can readily envision various other means and / or structures for performing the functions and / or obtaining one or more of the results and / or advantages described herein, and each of such variations and / or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications in which the teachings of the present invention are used. Those skilled in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is therefore to be understood that the above-described embodiments have been presented by way of example only, and that within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the inventive scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0051]

[0056] The above-described embodiments may be implemented in any of a number of ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code or instructions may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Furthermore, the instructions or software code may be stored in at least one non-transitory computer-readable storage medium.

[0052]

[0057] Also, a computer or smartphone utilized to execute software codes or instructions through its processor may have one or more input / output devices. These devices may be used, among others, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output, and a speaker or other sound generating device for audible presentation of output. Examples of input devices that may be used for a user interface include a keyboard, and pointing devices such as a mouse, a touchpad, and a discretization tablet. As another example, a computer may receive input information through voice recognition or in other audible formats.

[0053]

[0058] Such computers or smartphones may be interconnected by one or more networks of any suitable form, including enterprise networks, and local or wide area networks, such as an Intelligent Network (IN) or the Internet. Such networks may be based on any suitable technology and may operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0054]

[0059] The various methods or processes outlined herein may be coded as software / instructions executable on one or more processors employing any one of a variety of operating systems or platforms. Furthermore, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0055]

[0060] In this regard, the various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memory, USB flash drives, SD cards, circuitry in a field programmable gate array or other semiconductor device, or other non-transitory or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the present disclosure discussed above. The one or more computer-readable media may be transportable such that the one or more programs stored thereon can be loaded onto one or more different computers or other processors to implement the various aspects of the present disclosure discussed above.

[0056]

[0061] The terms "program" or "software" or "instructions" are used herein in a general sense to refer to any type of computer code or set of computer-executable instructions that may be employed to program a computer or other processor to implement various aspects of the embodiments discussed above. It should further be appreciated that, according to one aspect, one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.

[0057]

[0062] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0058]

[0063] Also, the data structures may be stored in the computer-readable medium in any suitable form. For simplicity of explanation, the data structures may be depicted as having fields that are related through their locations in the data structure. Such relationships may similarly be achieved by allocating storage of the fields with locations in the computer-readable medium that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information in the fields of the data structure, including through the use of pointers, tags, or other mechanisms that establish relationships between data elements.

[0059]

[0064] All definitions, as defined and used herein, are understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0060]

[0065] "Logic" as used herein includes, but is not limited to, hardware, firmware, software, and / or combinations of each for performing a function or action and / or causing a function or action from another logic, method, and / or system. For example, based on a desired application or need, logic may include software controlled microprocessors, discrete logic such as processors (e.g., microprocessors), application specific integrated circuits (ASICs), programmed logic devices, memory devices containing instructions, electrical devices having memory, etc. Logic may include one or more gates, combinations of gates, or other circuit components. Logic may also be fully comprised as software. When multiple logics are described, it may be possible to integrate the multiple logics into one physical logic. Similarly, when a single logic is described, it may be possible to distribute the single logic among multiple physical logics.

[0061]

[0066] Furthermore, the logic presented herein for accomplishing the various methods of this system may be directed to improvements of existing computer-centric or Internet-centric technologies that may not have previous similar versions. The logic may provide specific functionality directly related to structures that address and solve some of the problems identified herein. The logic may also provide significantly more advantages to solve these problems by providing exemplary inventive concepts as specific logical structures and harmonic functions of methods and systems. Furthermore, the logic may also provide specific computer-implemented rules that improve existing technological processes. The logic provided herein extends beyond simply collecting data, analyzing information, and displaying results. Furthermore, parts or all of this disclosure may rely on underlying equivalences derived from specific configurations of equipment or components recited herein. Thus, parts of this disclosure are not directed to abstract concepts as they relate to specific configurations of components. Furthermore, this disclosure and the appended claims present teachings that involve more than the performance of well-understood routine conventional activities previously known to the industry. In some of the methods or processes of the present disclosure, which may incorporate some aspect of a natural phenomenon, the process or method steps are additional features that are new and useful.

[0062]

[0067] The articles "a" and "an" as used herein in the specification and claims should be understood to mean "at least one" unless expressly indicated to the contrary. The phrase "and / or" as used herein in the specification and claims should be understood to mean "one or both" of the elements so conjoined (if at all), i.e., elements that are jointly present in some cases and disjunctively present in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so conjoined. Other elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to "A and / or B," when used with open-ended language such as "comprising," may in one embodiment refer only to A (possibly including elements other than B), in another embodiment may refer only to B (possibly including elements other than A), in yet another embodiment may refer to both A and B (possibly including other elements), and so forth. "Or" as used herein in the specification and claims should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as being inclusive, i.e., including at least one of the elements of the list, but also including two or more of them, and possibly additional unlisted items. Only terms clearly indicated to the contrary, such as "only one of" or "exactly one of," or "consisting of" when used in the claims, will refer to the inclusion of exactly one element of the elements of the list.In general, the term "or" as used herein should only be construed as indicating exclusive alternatives (i.e., "one or the other, but not both") when followed by terms of exclusivity such as "either," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0063]

[0068] It should be understood that, as used herein in the specification and claims, the phrase "at least one" in reference to a list of one or more elements means at least one element selected from any one or more of the elements in the list of elements, but does not necessarily include at least one of every element specifically listed in the list of elements, nor does it necessarily exclude any combination of elements in the list of elements. This definition also allows for elements other than those specifically identified in the list of elements to which the phrase "at least one" refers, may in some cases be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one A, optionally including more than one, but with no B (and optionally including elements other than B), in another embodiment to at least one B, optionally including more than one, but with no A (and optionally including elements other than A), in yet another embodiment to at least one A, optionally including more than one, and at least one B, optionally including more than one (and optionally including other elements), etc.

[0064]

[0069] When a feature or element is referred to herein as being "on" another feature or element, it may be directly on the other feature or element, or intervening features and / or elements may also be present. In contrast, when a feature or element is referred to as being "directly on" another feature or element, there are no intervening features or elements. Also, when a feature or element is referred to as being "connected," "attached," or "coupled" to another feature or element, it will be understood that it may be directly connected, attached, or coupled to the other feature or element, or there may be intervening features or elements. In contrast, when a feature or element is referred to as being "directly connected," "directly attached," or "directly coupled" to another feature or element, there are no intervening features or elements. Although described or illustrated with respect to one embodiment, the features and elements so described or illustrated may be applied to other embodiments. Additionally, it will be appreciated by those skilled in the art that a reference to a structure or feature disposed "adjacent to" another feature may have portions that overlap or underlie the adjacent feature.

[0065]

[0070] Spatially relative terms such as "below," "below," "lower," "above," "upper," "on," "behind," "in front of," and the like may be used herein to facilitate description of the relationship of an element or feature shown in a figure to another element or feature. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation shown in the figure. For example, if the device in the figure were inverted, an element described as "below" or "below" another element or feature would then be oriented "above" the other element or feature. Thus, the exemplary term "below" can encompass both an above and below orientation. The device may be oriented in other ways (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. Similarly, terms such as "upward," "downward," "vertical," "horizontal," "lateral," "transverse," "longitudinal," and the like are used herein for descriptive purposes only, unless specifically indicated otherwise.

[0066]

[0071] The terms "first" and "second" may be used herein to describe various features / elements, but these features / elements should not be limited by these terms unless the context indicates otherwise. These terms may be used to distinguish one feature / element from another. Thus, a first feature / element discussed herein may be referred to as a second feature / element, and similarly, a second feature / element discussed herein may be referred to as a first feature / element, without departing from the teachings of the present invention.

[0067]

[0072] An embodiment is an implementation or example of the present disclosure. References herein to "an embodiment," "one embodiment," "some embodiments," "one particular embodiment," "exemplary embodiment," or "other embodiments," etc., mean that a particular feature, structure, or characteristic described with respect to an embodiment is included in at least some embodiments of the invention, but not necessarily in all embodiments of the invention. The various appearances of "an embodiment," "one embodiment," "some embodiments," "one particular embodiment," "exemplary embodiment," or "other embodiments," etc., are not necessarily all referring to the same embodiment.

[0068]

[0073] When the specification states that a component, feature, structure, or characteristic "may," "might," or "could" be included, that particular component, feature, structure, or characteristic is not required to be included. When the specification or claims refer to "a" or "an" element, it does not mean that there is only one element. When the specification or claims refer to "additional" elements, it does not exclude that there are two or more additional elements.

[0069]

[0074] When used in this specification and claims, including when used in the examples, unless expressly specified otherwise, all numbers may be read as if preceded by the words "about" or "approximately" even if the words do not explicitly appear. The phrases "about" or "approximately" may be used when describing a size and / or location to indicate that the stated value and / or location is within a reasonable expected range of values ​​and / or locations. For example, a numerical value may have a value that is + / -0.1% of the stated value (or range of values), + / -1% of the stated value (or range of values), + / -2% of the stated value (or range of values), + / -5% of the stated value (or range of values), + / -10% of the stated value (or range of values), etc. Any numerical ranges recited herein are intended to include all subranges subsumed therein.

[0070]

[0075] Furthermore, methods embodying the present disclosure may be performed in a sequence different from that described herein. Thus, the sequence of the method should not be read as limiting unless expressly stated. It is recognizable that performing some of the steps of the method in a different order may achieve similar results.

[0071]

[0076] In the claims, as well as in the specification above, all transitional phrases such as "comprising," "including," "carrying," "having," "including," "involving," "holding," "consisting of," and the like, are understood to be open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" are closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures.

[0072]

[0077] In the above description, certain terms have been used for brevity, clarity, and understanding. Such terms are used for purposes of description and are intended to be broadly interpreted, so that no unnecessary limitations should be implied therefrom beyond the requirements of the prior art.

[0073]

[0078] Moreover, the description and illustration of the various embodiments of the present disclosure are examples, and the present disclosure is not limited to the exact details shown or described.

[0074]

[0079] The computer readable media described herein may be data storage devices or units such as magnetic disks, magneto-optical disks, optical disks, or flash drives. Additionally, it will be appreciated that the term "memory" herein is intended to encompass various types of suitable data storage media, whether permanent or transitory, such as transient electronic memory, non-transient computer readable media, and / or computer writeable media.

[0075]

[0080] Although various embodiments of the present invention have been described in detail, it is apparent that various modifications and alterations of those embodiments will occur and will be readily apparent to those skilled in the art. However, it should be clearly understood that such modifications and alterations are within the scope and spirit of the present invention, as set forth in the appended claims. Moreover, the invention described herein is capable of other embodiments and can be practiced or carried out in various other related ways. Furthermore, it should be understood that the phraseology and terminology used herein is for the purpose of description and should not be considered as limiting. The use of "including," "comprising," or "having" and variations thereof herein is intended to encompass the items listed thereafter and their equivalents as well as additional items, but only the terms "consisting of" and "consisting only of" should be interpreted in a limiting sense.

[0076]

[0081] The above description of the embodiments of the present disclosure has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the present disclosure be limited not by this detailed description, but rather by the claims appended hereto.

[0077]

[0082] Several implementations have been described. It will be understood, however, that various modifications may be made without departing from the scope of the present disclosure. Although operations are illustrated in the figures in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequence shown, or that all of the illustrated operations be performed, to achieve desirable results.

[0078]

[0083] While the principles of the present disclosure have been described herein, it should be understood by those skilled in the art that the present specification is made only as an example, and not as a limitation on the scope of the present disclosure. In addition to the exemplary embodiments shown and described herein, other embodiments are contemplated within the scope of the present disclosure. Modifications and substitutions made by those skilled in the art are deemed to fall within the scope of the present disclosure. The invention as described in the claims of the original application is set forth below. [C1] 1. A method for removal of burn-in data from a full motion video image, comprising: identifying one or more burned overlay areas in the full motion video image; creating a mask of said one or more burned overlay areas, said mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the dilated mask to a pixel replacement section, wherein the portion comprises the one or more burned overlay areas and a plurality of neighboring pixels proximate the one or more burned overlay areas; generating matched intensities for the pixels in the portion of the dilated mask based on a plurality of neighboring pixels; replacing the one or more burned overlay areas by filling the pixels in the one or more burned overlay areas with an adapted intensity to form a resultant image; A method comprising: [C2] The method of C1, further comprising subjecting the obtained images to video processing to obtain one or more detections. [C3] The method according to claim C1, wherein the obtained images are prepared for a video processing step in real time. [C4] The method of claim C1, wherein the mask is created using a color space decomposition method. [C5] The method according to C4, in which the mask generated via color space decomposition begins by converting the RGB image to the YCbCr color space. [C6] The method of C1, wherein the automatic replacement of one or more burned areas with matched pixels is via inpainting and ordinal filtering. [C7] The method of C6, wherein inpainting is used in a central portion of the full motion video image and ordinal filtering is used on the periphery of the full motion video image. [C8] The method according to C1, in which a binary mask is created. [C9] The method of C8, wherein the binary mask is provided to the inpainting algorithm. [C10] 1. A computer program product comprising one or more non-transitory machine-readable media having instructions encoded thereon that, when executed by one or more processors, cause a process for removing burn-in from full motion video imagery to be performed, the process comprising: identifying one or more burned overlay areas in the full motion video image; creating a mask of said one or more burned overlay areas, said mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the mask to an inpainting algorithm; generating matched intensities for the pixels in the mask based on neighboring unmasked pixels; removing the one or more burned overlay areas by filling the pixels in the mask with an adapted intensity to form a resultant image; subjecting the obtained images to video processing to obtain one or more detections; A computer program product comprising: [C11] The computer program product of C9, wherein the obtained images are prepared for a video processing step in real time. [C12] The computer program product of C9, wherein the mask is created using a color space decomposition method. [C13] The computer program product of C11, wherein the mask generated via color space decomposition begins by converting the RGB image to a YCbCr color space. [C14] The computer program product of C9, wherein the automatically replacing one or more burned areas with matched pixels is via inpainting and sequential filtering. [C15] The computer program product of C13, wherein inpainting is used in a central portion of the full motion video image and ordinal filtering is used on a periphery of the full motion video image. [C16] The computer program product of C9, wherein a binary mask is created. [C17] The computer program product of C15, wherein the binary mask is provided to the inpainting algorithm. [C18] 1. A method for removing burn-in from a full motion video image, comprising: identifying one or more burned overlay areas in the full motion video image; creating a mask of said one or more burned overlay areas, said mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the mask to an inpainting algorithm; creating adapted intensities for the pixels in the mask based on neighboring unmasked pixels, and wherein inpainting is used in a central portion of the full motion video image and ordinal filtering is used on a periphery of the full motion video image to create the adapted intensities. removing the one or more burned overlay areas by filling the pixels in the mask with an adapted intensity to form a resultant image; subjecting said obtained images to a video processing step in order to obtain one or more detections; A method comprising:

Claims

1. 1. A method for removal of burn-in data from a full motion video image, comprising: Identifying one or more burned overlay areas in the full motion video image; creating a mask of the one or more burned overlay areas, the mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the dilated mask to a pixel replacement section, wherein the portion comprises the one or more burned overlay areas and a plurality of neighboring pixels proximate the one or more burned overlay areas; generating matched intensities for the pixels in the portion of the dilated mask based on the neighboring pixels; replacing the one or more burned overlay areas by filling the pixels in the one or more burned overlay areas with the adapted intensity to form a resultant image; A method comprising:

2. The method of claim 1 , further comprising subjecting the obtained images to video processing to obtain one or more detections.

3. The method of claim 1 , wherein the acquired images are prepared for a video processing step in real time.

4. The method of claim 1 , wherein the mask is created using a color space decomposition method.

5. The method of claim 4 , wherein the mask generated via color space decomposition begins by converting the RGB image to a YCbCr color space.

6. The method of claim 1 , wherein automatically replacing one or more burned areas with matched pixels is via inpainting and sequential filtering.

7. The method of claim 6, wherein the inpainting is used in a central portion of the full-motion video image and the sequential filtering is used on the periphery of the full-motion video image.

8. The method of claim 1 , wherein a binary mask is created.

9. The method of claim 8 , wherein the binary mask is fed to an inpainting algorithm.

10. 1. A computer program comprising one or more non-transitory machine-readable media having instructions encoded thereon, the instructions, when executed by one or more processors, causing a process for removal of burn-in from full motion video imagery to be performed, the process comprising: Identifying one or more burned overlay areas in the full motion video image; creating a mask of the one or more burned overlay areas, the mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the mask to an inpainting algorithm; generating matched intensities for the pixels in the mask based on neighboring unmasked pixels; removing the one or more burned overlay areas by filling the pixels in the mask with the adapted intensity to form a resultant image; subjecting the obtained images to video processing to obtain one or more detections; A computer program comprising:

11. The computer program of claim 10 , wherein the obtained images are prepared for a video processing step in real time.

12. The computer program product of claim 10 , wherein the mask is created using a color space decomposition method.

13. 13. The computer program product of claim 12, wherein the mask generated via color space decomposition begins by converting the RGB image to a YCbCr color space.

14. The computer program product of claim 10 , wherein automatically replacing one or more burned areas with matched pixels is via inpainting and sequential filtering.

15. The computer program of claim 14, wherein the inpainting is used in a central portion of the full motion video image and the sequential filtering is used on the periphery of the full motion video image.

16. The computer program product of claim 10 , wherein a binary mask is created.

17. The computer program product of claim 16 , wherein the binary mask is provided to the inpainting algorithm.

18. 1. A method for removing burn-in from a full motion video image, comprising: Identifying one or more burned overlay areas in the full motion video image; creating a mask of the one or more burned overlay areas, the mask comprising a plurality of pixels; Expanding the mask; and providing at least a portion of the mask to an inpainting algorithm; creating adapted intensities for the pixels in the mask based on neighboring unmasked pixels, and wherein inpainting is used in a central portion of the full motion video image and ordinal filtering is used on a periphery of the full motion video image to create the adapted intensities. removing the one or more burned overlay areas by filling the pixels in the mask with the adapted intensity to form a resultant image; subjecting said obtained images to a video processing step in order to obtain one or more detections; A method comprising:

Images