Image processing method

Abstract

The invention relates to a method for processing a video stream of images captured by a color camera (2) on board of a motor vehicle (1) and used by a computer (3) to detect a priority vehicle (4), the method comprising the following steps: - a step (100) of acquisition of a sequence of images; for each image of the sequence of images: - a step (200) of colorimetric segmentation by thresholding, making it possible to detect colored lighted zones; - a step (300) of tracking of each segmented lighted zone, - a step (400) of colorimetric classification of each segmented lighted zone; - a step (500) of frequency analysis of each segmented lighted zone, making it possible to determine the flickering characteristic of said zone; - a step (700) of calculation of a global confidence index for each image of the sequence of images, making it possible to declare a lighted zone as a rotating beacon.

Description

Technical Field

[0001] This invention relates to image processing methods, and in particular to image processing methods for detecting emergency vehicles. Background Technology

[0002] Today, it is known to equip motor vehicles with driver assistance systems, commonly referred to as ADAS (“Advanced Driver Assistance Systems”). Such systems include imaging devices such as cameras mounted on the vehicle, which enable the generation of a series of images representing the vehicle's surroundings. For example, a camera mounted at the rear of the vehicle allows for the capture of images of the surrounding environment behind the vehicle, particularly vehicles behind. These images are then processed by a processing unit for purposes that assist the driver, such as by detecting obstacles (pedestrians, stopped vehicles, objects on the road, etc.) or by estimating the time of collision with an obstacle. Therefore, the information provided by the images acquired by the cameras must be reliable and relevant enough for the system to assist the driver.

[0003] In particular, most international laws and regulations stipulate that drivers must not obstruct the passage of priority vehicles (also known as emergency vehicles, such as fire trucks, ambulances, police cars, etc.) and should facilitate their passage. Therefore, ADAS systems should be able to identify such priority vehicles, especially when their lights (gyrophare) are activated, to avoid hindering their operation.

[0004] In current ADAS systems that include cameras that capture images of the front or rear of a vehicle, priority vehicles are detected in the same way as other standard (non-priority) vehicles. These systems typically implement a combination of machine learning and geometric perception methods. Images from these cameras are processed to extract bounding boxes around any type of vehicle (private vehicle, truck, bus, motorcycle, etc.), including emergency vehicles, making it impossible for existing ADAS systems to reliably distinguish between priority vehicles and standard vehicles behind them.

[0005] In addition, these systems suffer from partial or complete temporal occlusion of images captured by cameras that relate to the fact that priority vehicles do not need to follow regular driving rules but are able to weave in and out of lanes, shorten safe distances, or travel between two lanes, making existing systems unsuitable for such behaviors and situations.

[0006] Furthermore, priority or emergency vehicles are difficult to detect because they come in a wide variety of types. These vehicles are characterized by their LED or bulb-style flashers, which can be fixed or rotating, come in various colors, and are arranged in various ways. For example, some vehicles have a single flasher, while others have pairs, and still others have poles with more than two flashers, and so on. This diversity makes it more difficult for existing ADAS systems to reliably detect priority vehicles.

[0007] This detection problem is exacerbated in scenarios that also include the headlights and taillights of other vehicles in the surrounding environment behind the vehicle, as well as all other lights, and may increase the difficulty of detecting the flashing lights of the priority vehicle. Summary of the Invention

[0008] Therefore, the present invention proposes an image processing method for quickly and reliably detecting priority vehicles, regardless of the type of priority vehicle and its driving conditions, particularly by detecting the lights (flashlights) of these vehicles.

[0009] According to the present invention, this objective is achieved by a method for processing a video stream of images captured by at least one color camera mounted in a motor vehicle, the images being used by a computer mounted in the vehicle to detect priority vehicles in the environment surrounding the vehicle, the at least one camera being oriented towards the rear of the vehicle, the method being characterized by comprising the following steps:

[0010] - Steps for acquiring image sequences;

[0011] For each image in the image sequence:

[0012] - A segmentation step based on threshold (seuillage) for colorimetric segmentation enables the detection of colored emitting areas in the image that may be from a strobe lamp.

[0013] - Tracking steps for each segmented luminous region, thereby associating each segmented luminous region in the segmentation step with a predicted luminous region having the same color;

[0014] - A classification step that uses a pre-trained classifier to perform colorimetric classification of each segmented luminous region;

[0015] - A frequency analysis step that performs frequency analysis on each segmented light-emitting region enables the determination of the flicker characteristics of the segmented light-emitting region;

[0016] - The calculation steps for the overall confidence index of each image in the image sequence enable the declaring of a segmented luminous region as a strobe light.

[0017] Therefore, the method according to the invention enables reliable detection of the flashing lights of emergency vehicles, regardless of brightness and weather conditions, and the detection can reach a distance of 150 meters.

[0018] According to one embodiment, in the segmentation step, a predefined segmentation threshold is used to segment the luminescent region according to four categories:

[0019] -red,

[0020] -orange color,

[0021] - Blue, and

[0022] -Purple.

[0023] According to one embodiment, after the segmentation step, the method further includes a filtering step, referred to as a post-segmentation step, which enables filtering of the results from the segmentation step based on predetermined position criteria and / or size criteria and / or color criteria and / or intensity criteria. This filtering step enables the reduction of false detections.

[0024] According to one embodiment, the post-segmentation step includes a dimensional filtering sub-step, wherein luminous regions located in image portions far from the horizon and shadow points and having a size smaller than a predetermined dimensional threshold are filtered out. This step enables the exclusion of candidates corresponding to objects perceived by the camera as corresponding to measurement noise.

[0025] According to one embodiment, the post-segmentation step includes a sub-step of filtering out luminous regions whose size is greater than a predetermined dimensional threshold and whose luminous intensity is less than a predetermined luminous intensity threshold. This step enables the exclusion of candidates that, although near a vehicle, do not possess the luminous intensity required to function as a strobe light.

[0026] According to one embodiment, the post-segmentation step includes a location filtering sub-step, wherein luminous regions located below a horizontal line defined on the image in the image sequence are filtered out. This step enables the exclusion of candidates corresponding to the headlights of following vehicles.

[0027] According to one embodiment, the post-segmentation steps include a filtering sub-step that filters the segmented luminous regions according to a directional color threshold. This specific filtering enables more accurate color filtering. For example, for blue, a large number of false positives are filtered out when detecting luminous regions classified as blue, because the white light emitted by the headlights of following vehicles may be perceived as blue by the camera.

[0028] According to one embodiment, the method further includes a second segmentation step, in which, at the end of the tracking step, a second segmentation is performed on each segmented luminescent region for which no association was found.

[0029] According to one embodiment, the second segmentation step includes:

[0030] - The first sub-step involves relaxing the segmentation threshold and re-segmenting and tracking each image in the image sequence using these relaxed new segmentation thresholds, where the segmentation threshold corresponds to the color of the segmented luminous region.

[0031] - If no association is found at the end of the first sub-step, proceed to the second sub-step, where the segmentation threshold is modified to correspond to the segmentation threshold for white.

[0032] This step allows for the identification of the segmented luminous areas detected during segmentation. In effect, this final verification ensures that a false detection is indeed a false detection and not, for example, the headlights of a vehicle behind.

[0033] According to one embodiment, in the frequency analysis step, the flicker frequency of each segmented light-emitting region is compared with a first frequency threshold and a second frequency threshold greater than the first frequency threshold, both of which are predetermined. Segmented light-emitting regions are filtered out under the following conditions:

[0034] - If the flashing frequency of the segmented light-emitting area is less than the first frequency threshold, the segmented light-emitting area is considered to be non-flickering or flickering weakly, and therefore is not considered a rotating flashing light;

[0035] - The flashing frequency of the segmented light-emitting area is greater than the second frequency threshold, so the segmented light-emitting area is also considered not to be a rotating flash.

[0036] According to one embodiment, the first frequency threshold is equal to 1 Hz, and the second frequency threshold is equal to 5 Hz.

[0037] According to one embodiment, the method further includes a direction analysis step of performing direction analysis on each segmented light-emitting region, enabling the determination of the movement of the segmented light-emitting regions.

[0038] According to one embodiment, if the movement direction obtained in the direction analysis step allows for the conclusion that the segmented light-emitting area is filtered out:

[0039] - The segmented luminous area remains stationary relative to the vehicle;

[0040] - The segmented luminous area is far away from the vehicle.

[0041] The present invention also relates to a computer program product including instructions, which, when implemented by a computer, are used to perform a method comprising the following steps:

[0042] - Steps for acquiring image sequences;

[0043] For each image in the image sequence:

[0044] - A segmentation step based on threshold colorimetric segmentation enables the detection of colored emitting areas in the image that may be from a strobe lamp.

[0045] - Tracking steps for each segmented luminous region, thereby associating each segmented luminous region in the segmentation step with a predicted luminous region having the same color;

[0046] - A classification step that uses a pre-trained classifier to perform colorimetric classification of each segmented luminous region;

[0047] - A frequency analysis step is performed on each segmented luminous region to determine the flicker characteristics of the segmented luminous region;

[0048] - The calculation steps for the overall confidence index of each image in the image sequence enable the declaration of a segmented luminous region as a strobe light.

[0049] The present invention also relates to a vehicle including at least one color camera facing the rear of the vehicle and capable of acquiring a video stream of images of the surrounding environment behind the vehicle, and at least one computer configured to implement:

[0050] - Steps for acquiring multiple images;

[0051] For each image in the image sequence:

[0052] - A segmentation step based on threshold colorimetric segmentation enables the detection of colored emitting areas in the image that may be from a strobe lamp.

[0053] - Tracking steps for each segmented luminous region, thereby associating each segmented luminous region in the segmentation step with a predicted luminous region having the same color;

[0054] - A classification step that uses a pre-trained classifier to perform colorimetric classification of each segmented luminous region;

[0055] - A frequency analysis step is performed on each segmented luminous region to determine the flicker characteristics of the segmented luminous region;

[0056] - The calculation steps for the overall confidence index of each image in the image sequence enable the declaration of a segmented luminous region as a strobe light. Attached Figure Description

[0057] Other features, details, and advantages will become apparent by reading the following detailed description and studying the accompanying drawings, among which:

[0058] Figure 1This is a schematic representation of the vehicle and the priority vehicle according to the present invention.

[0059] Figure 2 An example implementation of the method according to the present invention is shown.

[0060] Figure 3 An embodiment of the post-segmentation step of the method according to the present invention is shown.

[0061] Figure 4 An embodiment of the second segmentation step according to the method of the present invention is shown.

[0062] Figure 5a The first image in an image sequence processed according to the method of the present invention is shown.

[0063] Figure 5b The second image in an image sequence processed according to the method of the present invention is shown.

[0064] Figure 5c The third image in an image sequence processed according to the method of the present invention is shown.

[0065] Figure 6 The state machine in a hysteresis form is shown.

[0066] Figure 7 Another state machine in a hysteresis form is shown.

[0067] Figure 8 The color space (U, V) is shown. Detailed Implementation

[0068] Figure 1 A vehicle 1 is schematically shown, which is equipped with a color camera 2 facing the rear of the vehicle 1 and capable of acquiring images of the surrounding environment behind the vehicle 1, and at least one computer 3 configured to use the images acquired by the camera 2. Figure 1 In the field of view of camera 2, the emergency vehicle or priority vehicle 4 is located behind vehicle 1. It should be noted that... Figure 1 The relative positions between vehicle 1 and emergency vehicle 4 shown are by no means restrictive.

[0069] Priority vehicles are characterized by their luminous points 5 and 6, also known as swivel lights, which emit blue, red, or orange light. These colors are used for priority vehicles in all countries. Priority vehicles may include one or more swivel lights, the arrangement of which can vary depending on the number of swivel lights they are equipped with (a single swivel light, or a pair of separate swivel lights spaced apart, or multiple swivel lights aligned and close to each other) and the position of these swivel lights on the priority vehicle's body (on the roof of the priority vehicle, on the front bumper of the priority vehicle, etc.). There are also many types of swivel lights. They can be LED lights or lights using incandescent bulbs.

[0070] The flashing lights of priority vehicles are also defined by their flashing characteristics during the alternating on and off phases.

[0071] Now refer to Figures 2 to 8 The method according to the present invention will be described.

[0072] The method according to the invention includes an acquisition step 100 of acquiring an image sequence, the image sequence including, for example, a first image I1, a second image I2 following the first image I1, and a third image I3 following the second image I2. Such images I1, I2, and I3 are respectively in... Figure 5a , 5b As shown in 5c.

[0073] The method according to the invention includes a segmentation step 200 of colorimetric segmentation based on a threshold.

[0074] This segmentation step 200 enables the detection and segmentation of the luminous region Z in each image of the image sequence using a predefined segmentation threshold. L i. This colorimetric segmentation is based on four color categories:

[0075] -red,

[0076] -orange color,

[0077] - Blue, and

[0078] -Purple.

[0079] Purple is particularly useful for detecting certain special swivel flashes with different chromaticities. For example, a swivel flash using a bulb may appear blue to the naked eye, but may be perceived as purple by a camera.

[0080] To accommodate the high diversity of flashing lights on priority vehicles, the thresholds used in segmentation step 200 are expanded relative to thresholds traditionally used for identification, such as traffic lights. These segmentation thresholds are predefined for each color, for saturation, for luminous intensity, and for chromaticity.

[0081] Segmentation step 200 provides the segmented luminous area Z of the emergency vehicle. Li (when these illuminated areas are lit) in the image, including their position, intensity, and color.

[0082] like Figure 5a As shown, at the end of the colorimetric segmentation step 200, multiple colored luminous areas Z that may be the flashlights of the priority vehicle were detected in the first image I1. L 1. Z L 2. Z L 3. Z L 4. Z L 5 and Z L 6.

[0083] Expanding the threshold value used in the segmentation step allows for adaptation to the diversity of flashing lights on priority vehicles, thus enabling sensitivity to a wider range of hues. However, this expansion introduces significant noise in segmentation step 200.

[0084] To reduce the number of potential candidates for detecting priority vehicles' flashlights, in other words, to reduce the number of false positives, the method according to the invention includes a post-segmentation step 210. This post-segmentation step 210 enables the detection of vehicles based on a predetermined luminous area Z. L The positional standard of i in the considered image, and / or the luminous area Z L i's size standard and / or light-emitting area Z L i's color standard and / or luminous area Z L The strength standard of i is used for filtering.

[0085] refer to Figure 3 After segmentation, step 210 includes a dimensional filtering sub-step 211, in which luminescent regions Z with a size smaller than a predetermined dimensional threshold are filtered out. L i, for example, a size less than 20 pixels (so-called small size). This dimensional filtering is particularly performed on image portions far from the horizontal line H representing infinity and the vanishing point F, i.e., image edges. In other words, this filter enables the removal of small-sized luminous areas Z that are not frequently encountered. L The small luminescent area Z in the image portion of i L i. In fact, when there is a small luminous area Z in the image L When i, when these luminescent areas Z L When i is near the horizontal line H, it corresponds to light that is farther away from vehicle 1. Therefore, the small luminous area Z that is not near the horizontal line H... L Therefore, i will not correspond to the light of the surrounding environment that is farther away from vehicle 1, but rather to the measurement noise, and for this purpose, such a light-emitting area Z will be filtered out. L i. Therefore, Figure 5a The light-emitting area Z shown L 1 is this situation.

[0086] Furthermore, as mentioned earlier, the luminous areas corresponding to farther-away light are located near the horizontal line H and the disappearance point F of image I1. Therefore, when the size of the luminous area is smaller than a predetermined size threshold, luminous areas located too far from the horizontal line H and the disappearance point F, especially those located at the side edges of image I1, are also filtered out. Figure 5a The light-emitting area Z shown L 4 is this situation.

[0087] After segmentation, step 210 includes filter step 212: filtering out light-emitting regions Z. L I represents luminous areas whose size is greater than a predetermined threshold, for example, whose luminous intensity is less than a predetermined luminous intensity threshold when their size is greater than 40 pixels (e.g., luminous intensity less than 1000 lux). In fact, the luminous areas filtered out in sub-step 212, although their size in the image corresponds to being relatively close to the light from vehicle 1 in the rear scene captured by camera 2, do not possess the luminous intensity required to be candidates of interest as priority vehicle flashes. Low-intensity luminous areas Z that are close enough to vehicle 1 to have a so-called large-size segmentation in the image. L i can, for example, simply correspond to the reflection of sunlight on a medium.

[0088] The segmentation step 210 further includes a position filtering sub-step 213, in which the luminescent areas located below the horizontal line H are filtered out. Therefore, Figure 5a The light-emitting area Z shown L 2 and Z L 3 is this situation. In fact, this type of position of the luminous area (below the horizontal line H) is particularly characteristic of the headlights of following vehicles, rather than the flashing lights of priority vehicles located above the horizontal line H.

[0089] The segmentation step 210 may further include a sub-step 214 of filtering conflicting luminous regions. At least two luminous regions are considered to conflict when they are close to each other, intersect, or one is contained within the other. When such a conflict is observed, only one of the two luminous regions is retained, and the other is filtered out. The luminous region to be removed from the two conflicting regions is determined in a manner known per se, based on predetermined brightness, size, and color standards for the regions.

[0090] The segmentation step 210 includes a sub-step 215 of filtering the segmented luminous regions according to a directional color threshold.

[0091] refer to Figure 8 This shows a color space (U, V) ranging from 0 to 255. Within this color space, based on the threshold U... max U min V max V minThe value of determines each color: R for red, V for purple, B for blue, and O for orange. For example, for blue, it is U. max (B) U min (B) V max (B) V min (B).

[0092] This method of defining colors by minimum and maximum values ​​on the U and V axes produces rectangular color patches that cannot represent reality and are therefore unsuitable. Therefore, color filtering is needed for each color. Filtering by directional color thresholds is equivalent to adjusting the minimum and maximum values ​​on the U and V axes of the color space for each color patch. Thus, the risk of false detection is reduced due to the proximity of the color patches (the proximity of the U and V values ​​between colors in the color space (U, V)). For example, with this directional color filtering, blue (B) that is too close to violet (V), red (R) that is too close to orange (O), and so on.

[0093] This directional color filtering allows for a more refined definition of color, thereby filtering out a large number of false positives when detecting luminescent areas.

[0094] Therefore, after performing the above sub-steps of step 210 following the segmentation, only the following light-emitting region Z... L i is retained as a potential candidate for priority vehicle flashing lights:

[0095] - The size is large enough (greater than 20 pixels);

[0096] -Located above the horizontal line H;

[0097] - Sufficient light emission (luminous intensity greater than 1000 lux);

[0098] - Its colorimetry is guaranteed (with appropriate U and V values ​​to avoid false positives).

[0099] It should be noted that it is not always necessary to perform all the sub-steps to execute the method according to the invention, but rather only some of them (individually or in combination) may be retained depending on the complexity of the image being processed.

[0100] Step 300 is the tracking step, which tracks each luminous region Z detected in each image. L i. The luminous region Z segmented in the image during segmentation step 200 is calculated by computer 3 in a manner known to those skilled in the art. LThe expected position of i is determined and used to ensure that the light detected in image In does indeed correspond to the same segmented luminous region in the previous image In-1, which may have moved. The expected position of the luminous region is determined by prediction. The expected position of the segmented luminous region in the current image In is calculated based on the position of the luminous region in the previous image In-1 plus a vector corresponding to the movement of the luminous region between images In-2 and In-1, while taking into account the movement of vehicle 1.

[0101] Furthermore, as described below, the flashing lights of priority vehicles are particularly characterized by their flashing frequency. Now, in order to estimate the flashing frequency of the flashing lights, it is necessary to be able to estimate their brightness changes over time (alternating between lit and unlit phases). Segmentation step 200 only applies to these segmented luminous areas Z. L i corresponds to the phase when the flashlights are lit, providing us with the position, intensity, and color of these areas in the image. Tracking step 300 enables the association of flashlights in each image and also enables the extrapolation of their positions when they are in the off-light phase (lights off).

[0102] The tracking step 300 known in the prior art has a threshold adapted to the flashing characteristics of the strobe light, and specifically enables the association of the luminous areas of the strobe light in each image, and also enables the extrapolation of their positions when the strobe light is in the off stage (corresponding to the absence of a corresponding luminous area in the image).

[0103] In a manner known per se, the light-emitting region Z in each segmentation step 200 will be... L i and the predicted luminous region Z with the same color P i is associated with.

[0104] If the predicted luminous region Z is not found in tracking step 300... P For any associated segment of the luminous region Z of i L i. At the end of tracking step 300, the second segmentation step 310 is performed.

[0105] The second segmentation step 310 includes a first sub-step 311, in which the segmentation threshold is relaxed (in other words, the segmentation threshold is defined as less stringent and less filtering), and the segmentation step 200 and the tracking step 300 are re-executed for each image in the image sequence using these relaxed new segmentation thresholds. This step enables the detection of the predicted segmentation region Z. P The segmented luminous region Z in i L i.

[0106] If at the end of the first sub-step 311, the processed segmented luminescent region Z... L i and predicted luminous region Z PIf no correlation is found between i, then the second sub-step 312 is performed, where the segmentation threshold is modified to correspond to white. In effect, this final verification ensures that the false detection is indeed a false detection, and not the headlight of a vehicle behind.

[0107] The second sub-step 312 specifically enables the detection, for example, of the headlights of vehicles behind which the white light may contain blue light.

[0108] The method according to the invention then includes processing each light-emitting region Z L i. Classification steps for colorimetric classification 400.

[0109] This classification step 400 enables the selection of the luminescent region Z from the segmentation step 200. L i. For each color, (in the so-called offline pre-training step) a classifier is trained to distinguish between positive data (representing the flashlights to be detected) and negative data (representing all noise from segmentation step 200 that is not a flashlight, and therefore it is expected that they will not be detected, such as vehicle headlights or taillights, sunlight reflections, signal lights, etc.).

[0110] If the classifier cannot classify (recognize) the segmented luminescent region Z during the classification step... L If i, then the luminous region is filtered out.

[0111] Conversely, if the classifier identifies the luminous region Z... L If i is selected, it is retained as an important candidate for a rotating flash. At the end of classification step 400, candidate luminous regions Z are obtained. C The list of i, these candidate luminescent regions Z C i is characterized by the following parameters:

[0112] -Flickering state;

[0113] -Classification Confidence Index I CC ;

[0114] - Location in the image;

[0115] -color.

[0116] The flashing status is obtained by detecting the flickering of the rotating flashlight. This detection includes:

[0117] - Count the number of images as follows: where the strobe light is on and therefore the corresponding luminous area Z is detected. L 5 and Z L 6(reference Figure 5a Image I1),

[0118] - Count the number of images as follows: where the flashlight is off and therefore no corresponding luminous area Z is detected. L 5 and ZL 6(reference Figure 5b Image I2), and

[0119] - Count the number of images as follows: where the flashlight turns on again and therefore the corresponding luminous area Z is detected again. L 5 and Z L 6(reference Figure 5c Image I3).

[0120] The confidence index I is obtained by using the positive classification and blinking information (blinking state) of the classifier in step 400. CC According to one embodiment, after classification step 400, the luminescent region Z of each segment is updated. L The confidence index of i.

[0121] If the classification is positive, then the classification confidence index I of the image relative to time t-1 is calculated according to the following formula. CC Classification confidence index I of the image at update time t CC :

[0122]

Mathematical Formula 1

[0123] I CC (t)=I CC (t-1)+FA

[0124] Where FA is a predetermined increment factor.

[0125] If the classification is negative, then the classification confidence index I of the image relative to time t-1 is calculated according to the following formula. CC Classification confidence index I of the image at update time t CC :

[0126]

Mathematical Formula 2

[0127] I CC (t)=I CC (t-1)-FR

[0128] Where FR is a predetermined decreasing factor.

[0129] The location and color information are provided by classification step 200.

[0130] At the end of classification step 400, in order to determine the candidate luminescent region Z C Whether i might be a flashing light of an emergency vehicle, the method includes performing frequency analysis to calculate and threshold segmentation of the luminous area Z. L The frequency analysis steps for the flicker frequency of i and the calculation steps for the time integral of the classifier response are detailed below.

[0131] In step 500, for each segmented light-emitting region Z L Frequency analysis is performed to determine the segmented luminous region Z. L The flickering or non-flickering characteristics of i.

[0132] Advantageously, prior to step 500, the segmented light-emitting region Z is corrected. L The inconsistency that i may exhibit. Specifically, for the segmented luminescent region Z. L This correction is made based on the color, size, or intensity of i. If excessive color fluctuations are detected, for example, if the segmented luminous region Z... L If i changes from red to orange between two images, then the segmented luminous region Z is filtered out. L i. Or, if the segmented luminous region Z L If the size of i varies too much between images (e.g., a variation greater than 2), then the segmented luminous region Z is filtered out. L i.

[0133] Based on the detection of the on and off phases of the flashing light, which enables the determination of the flashing frequency, the known Fast Fourier Transform (FFT) allows the determination of the flashing frequency.

[0134] In frequency analysis step 500, each segmented luminescent region Z is... L The flicker frequency of i is related to the first frequency threshold S. F 1 and greater than the first frequency threshold S F The second frequency threshold S of 1 F 2. A comparison is made between these two predetermined thresholds. If the flicker frequency is less than the first frequency threshold S... F 1, then the segmented luminous region Z L i is considered to have no flicker, therefore it is not a swivel light and is filtered out. If the flicker frequency is greater than the second frequency threshold S... F 2, then the segmented luminous region Z L i was also considered not to be a swivel light and was filtered out.

[0135] This flicker frequency analysis allows for the filtering out of segmented light-emitting areas Z that are certain to remain constant, or that flicker too slowly for priority vehicles, or vice versa. L i. Therefore, the candidate frequencies retained as of interest are the flicker frequencies at the frequency threshold S. F 1 and S F The light-emitting area Z is divided between 2. L i.

[0136] According to one embodiment, the first frequency threshold S F 1 equals 1Hz, and the second frequency threshold S F2 equals 5 Hz. Furthermore, according to one example, the flashing lights on police cars and ambulances typically operate at frequencies between 60 and 240 FPM. FPM (flashes per minute) is a unit of measurement used to quantify the flashing frequency of a flashing light, corresponding to the number of cycles occurring per minute. Values ​​measured in FPM can be converted to Hertz by dividing by 60. In other words, for these priority vehicles, their frequencies are between 1 Hz and 6 Hz.

[0137] Frequency threshold S F 1 and S F These values ​​of 2 in particular enable:

[0138] - Adapted to the camera (in fact, visual persistence makes it impossible for us to see certain lights flashing, like the lights on the roof of a taxi, but the camera's resolution is able to detect it);

[0139] - To obtain more robust results when facing potential errors in the segmentation step 200.

[0140] In optional step 600, for each segmented light-emitting region Z L i performs directional analysis in order to segment the luminous region Z in the image sequence acquired by the camera. L The tracking of i determines the segmented luminous region Z. L i moves relative to vehicle 1. Therefore, if the segmented luminous area Z L i is moving away from vehicle 1 (in other words, if the segmented luminous area Z) L If i is approaching a vanishing point F or a horizontal line H in the image, then the segmented luminous area Z is filtered out. L i. If the segmented luminous region Z L i is stationary, or this is also possible. This step, in particular, enables the filtering out of such segmented luminescent regions Z. L i: This does indeed correspond to the flashing lights, but the priority vehicles move in the opposite direction to vehicle 1, so vehicle 1 does not have to consider these priority vehicles. This step also makes it possible to filter out the taillights of cars traveling in the opposite lane to vehicle 1's lane.

[0141] This step enables the fusion and segmentation of the light-emitting region Z. L i-related information is used to improve the performance of emergency vehicle flasher detection (true positives and false positives). In fact, this is achieved by segmenting the luminous area Z... L Studying i as a whole (in all images of the image sequence) rather than in isolation can reduce the false positive rate while maintaining a satisfactory detection rate.

[0142] In this method step, a set of segmented luminous regions Z are detected in the images of the image sequence. Li. These detected segmented luminescent regions Z L i is stored in the memory of computer 3 as a list, which includes each segmented light-emitting area Z. L The identifier associated with i and these segmented luminous areas Z L The parameters of i, for example:

[0143] - Its location in the image;

[0144] - Its dimensions;

[0145] - Its color;

[0146] -Its strength;

[0147] -Its flashing state;

[0148] -The classification confidence index I associated with it CC .

[0149] In this step, all detected and retained segmented luminescent regions Z L i is considered a potential emergency vehicle flasher. To determine whether an emergency vehicle flasher is present in the scene (the surrounding environment behind the vehicle, corresponding to the image sequence captured by camera 2), the method concludes with a step 700 of analyzing the scene, including calculating an overall confidence index I for each image in the image sequence for each of the colors red, orange, blue, and purple. CG According to one embodiment, the overall confidence index can be grouped by color. For example, the confidence index for purple can be combined with the confidence index for blue.

[0150] Therefore, an instantaneous confidence index I is calculated for each image in the image sequence and for each of the colors red, orange, blue, and purple. CI .

[0151] Illumination region Z based on segmentation of the current image in the image sequence L The parameters of i are used to calculate the instantaneous confidence. First, consider a flickering state and a sufficient classification confidence index I. CC The segmented light-emitting area Z L i.

[0152] To determine its classification confidence index I CC The segmented luminescent region Z is sufficient to be considered. L i, Figure 7 The state machine in a hysteresis form is shown.

[0153] Light-emitting region Z L The state of i is initialized (Ei) to the "OFF" state:

[0154] -If the classification confidence index ICC If the value exceeds the predetermined threshold C3, the state "off" will change to "on".

[0155] -If the classification confidence index I CC If the value is less than the predetermined threshold C4, the state changes from "on" to "off". The instantaneous confidence index I is calculated for an image and a color. CI One example could be the segmented luminous regions Z detected in all images within an image sequence for that color. L Classification confidence index I of i CC The sum. For example, for red:

[0156]

Mathematical Expression 3

[0157] Ici(red)=∑I CC (red).

[0158] Then, this instantaneous confidence is filtered over time to obtain an overall confidence index for each image in the image sequence and for each of the red, orange, blue, and purple colors, according to the following formula:

[0159]

Mathematical Expression 4

[0160] ICG(t)=(1-α)*ICG(t-1)+α*Ici

[0161] -Ici is the instantaneous confidence index of the color under consideration;

[0162] -α is a predetermined coefficient associated with color, and enables the determination of the instantaneous confidence index Ici within the overall confidence index I. CG The weights in the calculation, for example, the value of the coefficient α is between 0 and 1, and preferably between 0.02 and 0.15.

[0163] The higher the value of coefficient α, the higher the instantaneous confidence index Ici is relative to the overall confidence index I. CG The greater the weight in the calculation, the better.

[0164] According to one embodiment, the coefficient α is based on the segmented light-emitting region Z. L The parameter i varies. For example, the coefficient α:

[0165] - In multiple segmented light-emitting areas Z L i has a classification confidence index I greater than a predetermined threshold. CC Increase over time;

[0166] - This is reduced when Computer 3 has indicated the presence of an emergency vehicle in the scene, which allows for the segmentation of the luminous area Z. LThe results of this method remain stable and robust even in the event of transient detection loss in the detection chain of i, which may be caused by, for example, occlusion, excessive brightness, or excessive measurement noise.

[0167] According to one embodiment, the coefficient α is based on the segmented light-emitting region Z. L The position of i in the image varies. Specifically, the coefficient α:

[0168] -The image includes isolated, segmented luminous regions Z located below the horizontal line H. L The speed decreases in case i (in other words, the integration speed is lower);

[0169] - In multiple segmented light-emitting areas Z L i increases when it is above the horizontal line H (this corresponds to a high integral velocity).

[0170] According to one embodiment, the coefficient α is based on the segmented light-emitting region Z. L i varies with respect to their relative positions in the image. Specifically, if multiple segmented luminous regions Z... L If the i are aligned on the same straight line L, then the coefficient α increases.

[0171] Variations in parameter α allow for adjustment of the method's sensitivity, thereby adapting to the segmented luminescent region Z. L The parameter of i determines whether the event is detected faster or slower.

[0172] It is also possible to assign the luminous region Z to each segment of the current image in the image sequence based on other parameters. L Assign weights to i, for example:

[0173] -If the segmented luminous region Z L If the size of i is very small (in other words, its size is less than, for example, 20 pixels) and / or close to the horizontal line H, then the value of this weight decreases;

[0174] - If the segmented luminescent region emits weak light (with a luminescence intensity of, for example, less than 1000 lux), the value of this weight is reduced;

[0175] - If the color of the light is not distinct (e.g., in the case of saturated white light), the value of this weight is reduced;

[0176] -If the segmented luminous region Z L i (in terms of size, location, intensity, and color) closely resembles other similarly segmented luminous areas Z. L If i, then the value of that weight increases.

[0177] When multiple segmented light-emitting regions Z L When i has a strong correlation with location, brightness, and color, this weighting enables the acceleration of the overall confidence index I.CG The increase in [the overall confidence index I] is slowed down when the light size is small and the intensity is low. CG The increased density allows for a reduction in the false positive rate, although this results in slower detection of emergency vehicles at distant locations, which is acceptable.

[0178] Step 700 enables a significant reduction in the false positive rate while still maintaining a satisfactory detection rate for the emergency vehicle's flashing lights.

[0179] At the end of step 700, computer 3 can indicate the presence of an emergency vehicle in the surrounding environment behind vehicle 1 (corresponding to the image sequence acquired by camera 2), and in particular, declare the luminous area Z by means of a hysteresis threshold known per se. L The "i" is the emergency vehicle's flashing light.

[0180] Figure 6 The state machine in a hysteresis form is shown.

[0181] Light-emitting region Z L The state of i is initialized (Ei) to the "off" state:

[0182] - If the overall confidence index ICG is greater than the predetermined threshold C1, the state "off" will change to the state "on".

[0183] - If the overall confidence index ICG is greater than the predetermined threshold C2, the state "on" will change to the state "off".

[0184] The driver of the vehicle (in the case of an autonomous vehicle) or vehicle 1 can then take the necessary measures to facilitate rather than obstruct the movement of the emergency vehicle.

Claims

1. A method for processing a video stream of images captured by at least one color camera (2) mounted in a motor vehicle (1), the images being used by a computer (3) mounted in the vehicle to detect priority vehicles (4) in the environment surrounding the vehicle (1), the at least one camera being oriented toward the rear of the vehicle, the method being characterized in that it comprises the following steps: - Acquisition steps for acquiring image sequences (100); For each image in the image sequence: - A segmentation step (200) based on a threshold for colorimetric segmentation enables the detection of colored emitting regions (ZLi) in the image that may be the strobe lights (5, 6). - A tracking step (300) that tracks each segmented luminous region (ZLi) and associates each segmented luminous region in the segmentation step (200) with a predicted luminous region (ZPi) having the same color; - Classification step (400) to perform colorimetric classification of each segmented luminescent region (ZLi) using a pre-trained classifier. - A frequency analysis step (500) performs frequency analysis on each segmented luminescent region (ZLi) to determine the scintillation characteristics of the segmented region (ZLi); - The calculation step (700) calculates the overall confidence index (ICG) for each image in the image sequence, taking into account the classification confidence index from the classification step (400), so that a segmented luminous region (ZLi) can be declared as a strobe light.

2. The method according to claim 1, characterized in that, In the segmentation step (200), a predefined segmentation threshold is used to segment the luminescent region (ZLi) according to four categories: - red, - orange color, - Blue, and - Purple.

3. The method according to claim 1, characterized in that, Following the segmentation step (200), the method further includes a filtering step called a post-segmentation step (210), which enables the filtering of the results from the segmentation step (200) based on predetermined position criteria and / or size criteria and / or color criteria and / or intensity criteria.

4. The method according to claim 3, characterized in that, The segmentation step (210) includes a dimensional filtering sub-step (211), in which luminous regions (Z) located in the image portion far from the horizontal line (H) and the vanishing point (F) and having a size smaller than a predetermined dimensional threshold are filtered out. L i).

5. The method according to claim 3, characterized in that, The segmentation step (210) includes filtering out luminescent regions (Z) whose size is greater than a predetermined dimension threshold and whose luminous intensity is less than a predetermined luminous intensity threshold. L Sub-step (212) of i).

6. The method according to claim 3, characterized in that, The segmentation step (210) includes a position filtering sub-step (213), in which luminous regions (Z) located below the horizontal line (H) defined on the image in the image sequence are filtered out. L i).

7. The method according to claim 3, characterized in that, The post-segmentation step (210) includes targeting the segmented light-emitting region (Z). L i) The filtering sub-step (215) of filtering by directional color threshold.

8. The method according to any one of claims 1 to 7, characterized in that, It also includes a second segmentation step (310), which, at the end of the tracking step (300), identifies the luminescent region (Z) of each segment for which no association is found. L i) Perform a second segmentation.

9. The method according to claim 8, characterized in that, The second segmentation step (310) includes: - First sub-step (311), wherein the segmentation threshold is relaxed, and each image in the image sequence is re-segmented using these relaxed new segmentation thresholds in step (200) and tracking step (300), the segmentation threshold being a segmentation threshold corresponding to the color of the segmented luminous region, and - If no association is found at the end of the first sub-step, proceed to the second sub-step (312), where the segmentation threshold is modified to correspond to the segmentation threshold for white.

10. The method according to any one of claims 1 to 7, characterized in that, In the frequency analysis step (500), each segmented luminescent region (Z) is... L i) flicker frequency and first frequency threshold (S F 1) and greater than the first frequency threshold (S) F 1) Second frequency threshold (S) F 2) Compare these two frequency thresholds, both of which are predetermined. Filter out the segmented luminescent region (Z) under the following conditions: L i): - The flicker frequency of the segmented light-emitting area is less than the first frequency threshold (S). F 1) This makes the segmented light-emitting area considered to be non-flickering or only slightly flickering, and therefore not considered a rotating flash; - The flicker frequency of the segmented light-emitting area is greater than the second frequency threshold (S). F 2) This makes the light-emitting area also considered not to be a rotating flash.

11. The method according to claim 10, characterized in that, First frequency threshold (S) F 1) equals 1Hz, and the second frequency threshold (S) F 2) Equal to 5Hz.

12. The method according to any one of claims 1 to 7, characterized in that, It also includes the light-emitting area (Z) for each segment. L i) A directional analysis step (600) is performed to determine the movement of the segmented light-emitting area.

13. The method according to claim 12, characterized in that, If the direction of movement obtained in the direction analysis step (600) allows for the following conclusion, then the segmented luminous region (Z) is filtered out. L i): - Segmented light-emitting area (Z) L i) Relative to the vehicle (1), it is stationary; - Segmented light-emitting area (Z) L i) Far away from the vehicle (1).

14. A computer program product including instructions, said instructions being configured, when implemented by a computer, to perform a method comprising the following steps: - Acquisition steps for acquiring image sequences (100); For each image in the image sequence: - A segmentation step (200) based on threshold colorimetric segmentation enables the detection of colored emitting regions (Zi) in the image that may be the strobe lights (5, 6). L i) - Track the luminescent region of each segment (Z) L The tracking step (300) of i) is based on which the light-emitting region (Z) of each segment in the segmentation step (200) is segmented. L i) and the predicted luminous region (Z) with the same color P i) Related; - Each segmented luminous region (Z) is analyzed using a pre-trained classifier. L i) Classification steps for colorimetric classification (400); - For each segmented light-emitting region (Z) L i) A frequency analysis step (500) is performed to determine the flicker characteristics of the segmented light-emitting region; - Calculate the overall confidence index (I) for each image in the image sequence. CG The calculation step (700) considers the classification confidence index from the classification step (400), enabling a segmented luminescent region (Z) to be classified. L i) Declare it as a rotating flashlight.

15. A vehicle (1), comprising at least one color camera (2) facing the rear of the vehicle and capable of acquiring a video stream of images of the surrounding environment behind the vehicle, and at least one computer (3), the computer (3) being configured to implement: - Acquisition steps for acquiring image sequences (100); For each image in the image sequence: - A segmentation step (200) based on threshold colorimetric segmentation enables the detection of colored emitting regions (Zi) in the image that may be the strobe lights (5, 6). L i) - Track the luminescent region of each segment (Z) L The tracking step (300) of i) is based on which the light-emitting region (Z) of each segment in the segmentation step (200) is segmented. L i) and the predicted luminous region (Z) with the same color P i) Related; - Each segmented luminous region (Z) is analyzed using a pre-trained classifier. L i) Classification steps for colorimetric classification (400); - For each segmented light-emitting region (Z) L i) A frequency analysis step (500) is performed to determine the flicker characteristics of the segmented light-emitting region; - Calculate the overall confidence index (I) for each image in the image sequence. CG The calculation step (700) considers the classification confidence index from the classification step (400), enabling a segmented luminescent region (Z) to be classified. L i) Declare it as a rotating flashlight.

Images