Image processing method to declare a segmented area as circumferential light

Description

technical field

[0001] The present invention relates to an image processing method, particularly for the detection of emergency vehicles. Previous technique

[0002] Nowadays, it is common to equip a motor vehicle with a driver assistance system commonly called ADAS (Advanced Driver Assistance System). Such a system typically includes an imaging device, such as a camera mounted on the vehicle, which generates a series of images representing the vehicle's surroundings. For example, a camera mounted at the rear of the vehicle can film the area behind it, including vehicles following behind. These images are then processed by a control unit to assist the driver, for example, by detecting obstacles (pedestrians, stationary vehicles, objects on the road, etc.) or by estimating the time before a collision with obstacles. The information provided by the images acquired by the camera must therefore be sufficiently reliable and relevant for the system to assist the driver.

[0003] In particular, most international legislation stipulates that a driver must not obstruct the passage of priority vehicles in operation (also known as emergency vehicles, such as fire trucks, ambulances, police cars, etc.) and must facilitate their movement. Therefore, it is essential that ADAS systems be able to recognize such priority vehicles, especially when their emergency lights (flashing lights) are activated, so as not to impede their response.

[0004] In current ADAS systems that use a camera to film the front or rear of the vehicle, priority vehicles are detected in the same way as other standard (non-priority) vehicles. These systems typically implement a combination of machine learning and geometric perception approaches. The images from these cameras are processed to extract bounding boxes around all types of vehicles (cars, trucks, buses, motorcycles, etc.), including emergency vehicles. Consequently, existing ADAS systems cannot reliably distinguish a priority following vehicle from a standard following vehicle.

[0005] Furthermore, these systems are subject to problems of partial or total temporal occlusion of the images acquired by the camera and related to the fact that a priority vehicle is not required to respect the classic traffic rules and may zigzag between lanes, reduce safety distances or travel between two lanes, so that existing systems are not adapted to such behaviors and circumstances.

[0006] Furthermore, detecting priority or emergency vehicles is difficult because there is a wide variety of priority vehicle types. These vehicles are characterized by their flashing lights, which can be LED or incandescent, fixed or rotating, and come in different colors and configurations. For example, some vehicles are equipped with a single flashing light, others with pairs, and still others with light bars containing more than two lights, and so on. This variability makes reliable detection of priority vehicles even more challenging for existing ADAS systems.

[0007] In addition, there are detection problems for scenes in which there are headlights and taillights of other vehicles and any other lights present in the rear environment of the vehicle which may increase the difficulty of detecting the flashing lights of priority vehicles known from patent EP2523173 A1. Presentation of the invention

[0008] The present invention therefore proposes an image processing method for the rapid and reliable detection of priority vehicles regardless of the type of priority vehicle and regardless of the conditions in which it travels, in particular by detecting the lights (rotating beacons) of these vehicles.

[0009] According to the invention, this goal is achieved through a method of processing a video stream of images captured by at least one color camera mounted in a motor vehicle, said images being used by a computer mounted in said vehicle to detect a priority vehicle located in the vehicle's vicinity, at least one camera being oriented towards the rear of the vehicle, said method being characterized in that it comprises the following steps: a step in acquiring a sequence of images; for each image in the image sequence: a colorimetric segmentation step by thresholding allowing the detection of colored luminous areas of the image likely to be flashing lights; a tracking step of each segmented luminous area whereby each luminous area segmented during the segmentation step is associated with a prediction luminous area of ​​the same color; a colorimetric classification step, by a previously trained classifier, of each segmented luminous area; a frequency analysis step of each segmented luminous area allowing to determine a flashing character of said segmented luminous area; a calculation step of an overall confidence index of each image of the image sequence, taking into account a classification confidence index from the colorimetric classification step, allowing to declare a segmented luminous area as being a flashing light.

[0010] The method according to the invention thus makes it possible to reliably detect the flashing lights of an emergency vehicle regardless of lighting and weather conditions, up to a distance of 150 meters.

[0011] According to one example implementation, during the segmentation step, predefined segmentation thresholds are used to segment the light areas into four categories: the colors red, orange, blue, and purple.

[0012] In one embodiment, after the segmentation step, the process includes a further filtering step, known as post-segmentation, which filters the results from the segmentation step. This post-segmentation step is performed according to predetermined criteria of position, size, color, and / or intensity. This filtering step reduces false detections.

[0013] In one embodiment, the post-segmentation step includes a dimensional filtering substep in which bright areas located in parts of the image far from a horizon line and a vanishing point, and smaller than a predetermined dimensional threshold, are filtered out. This step eliminates candidates corresponding to objects perceived by the camera that represent measurement noise.

[0014] In one embodiment, the post-segmentation step includes a substep for filtering out luminous areas that exceed a predetermined dimensional threshold but have a luminous intensity below a predetermined intensity threshold. This step eliminates candidates that, although close to the vehicle, do not have the required luminous intensity to be considered emergency beacons.

[0015] In one example, the post-segmentation step includes a positional filtering substep in which bright areas positioned below a defined horizon line on the image sequence are filtered out. This step eliminates candidates corresponding to the headlights of a following vehicle.

[0016] In one embodiment, the post-segmentation step includes, for a segmented light area, a substep of oriented chromatic thresholding filtering. This specific filtering allows for more precise color filtering. For example, for the color blue, it filters out a large number of false positives in the detection of light areas classified as blue, due to the fact that the white light emitted by the headlights of following vehicles can be perceived as blue by the camera.

[0017] According to one embodiment, the process further includes a second segmentation step, after the follow-up step, for each segmented light area for which no association has been found.

[0018] According to an example implementation, the second segmentation step includes: a first sub-step in which the segmentation thresholds are broadened and the segmentation and tracking steps are renewed for each image in the image sequence with these new broadened segmentation thresholds, the segmentation thresholds being those corresponding to the color of the segmented light area, and if at the end of this first sub-step no association has been found, a second sub-step in which the segmentation thresholds are modified so as to correspond to those of the color white.

[0019] This step confirms the segmented light zones (detected) during the segmentation process. This final check ensures that the false detection was indeed a false positive and not, for example, a headlight from a following vehicle.

[0020] According to one embodiment, during the frequency analysis step, the blinking frequency of each segmented light zone is compared to a first frequency threshold and a second frequency threshold higher than the first frequency threshold, predetermined, a segmented light zone being filtered if: its flashing frequency is below the first frequency threshold, so the segmented light area is considered to be non-flashing or weakly flashing and therefore not a rotating beacon; its flashing frequency is above the second frequency threshold, so the segmented light area is also considered not to be a rotating beacon.

[0021] According to one example implementation, the first frequency threshold is equal to 1 Hz and the second frequency threshold is equal to 5 Hz.

[0022] According to one embodiment, the process further includes a directional analysis step of each segmented light zone, enabling the determination of a displacement of said segmented light zone.

[0023] According to an example implementation, a segmented light zone is filtered if the direction of movement obtained during the directional analysis step allows us to conclude that: immobilization of the segmented light zone, relative to the vehicle; a moving away of the segmented light zone, relative to the vehicle.

[0024] The invention also relates to a computer program product, comprising instructions for implementing a method including: a step in acquiring a sequence of images; for each image in the image sequence: a colorimetric segmentation step by thresholding allowing the detection of colored luminous areas of the image likely to be flashing lights; a tracking step of each segmented luminous area whereby each luminous area segmented during the segmentation step is associated with a prediction luminous area of ​​the same color; a colorimetric classification step, by a previously trained classifier, of each segmented luminous area; a frequency analysis step of each segmented luminous area allowing the determination of a flashing character of the segmented luminous area; a calculation step of an overall confidence index of each image of the image sequence, taking into account a classification confidence index from the colorimetric classification step, allowing a luminous area to be declared as a flashing light, when implemented by a computer.

[0025] The invention also relates to a vehicle, comprising at least one rear-facing color camera capable of acquiring a video stream of images from the rear of the vehicle and at least one computer, the computer being configured to implement: a step of acquiring a plurality of images; for each image in the image sequence: a colorimetric segmentation step by thresholding, allowing the detection of colored luminous areas of the image likely to be flashing lights; a tracking step of each segmented luminous area whereby each luminous area segmented during the segmentation step is associated with a prediction luminous area of ​​the same color; a colorimetric classification step, by a previously trained classifier, of each segmented luminous area; a frequency analysis step of each segmented luminous area allowing to determine a flashing character of the segmented luminous area; a calculation step of an overall confidence index of each image of the image sequence, taking into account a classification confidence index from the colorimetric classification step, allowing to declare a segmented luminous area as being a flashing light. Brief description of the drawings

[0026] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: [ Fig. 1 ] There figure 1 is a schematic representation of a vehicle according to the invention and of a priority vehicle. Fig. 2 ] There figure 2 represents an example of implementing the process according to the invention. Fig. 3 ] There figure 3 illustrates an example of implementing the post-segmentation step of the process according to the invention. Fig. 4 ] There figure 4 illustrates an example of implementing the second segmentation step of the process according to the invention. Fig. 5a ] There figure 5a illustrates a first image of the image sequence processed by the method according to the invention. Fig. 5b ] There figure 5b illustrates a second image from the image sequence processed by the method according to the invention. Fig. 5c ] There figure 5cillustrates a third image from the image sequence processed by the method according to the invention. Fig. 6 ] There figure 6 illustrates a state machine in hysteresis form. Fig. 7 ] There figure 7 illustrates another state machine in the form of hysteresis. Fig. 8 ] There figure 8 illustrates a color space (U,V). Description of the implementation methods

[0027] On the figure 1 The diagram schematically represents a vehicle 1 equipped with a color camera 2, oriented towards the rear of the vehicle 1 and capable of acquiring images of the rear environment of the vehicle 1, and at least one computer 3 configured to process the images acquired by the camera 2. figure 1 An emergency vehicle, or priority vehicle, 4 is positioned behind vehicle 1, within the field of vision of camera 2. Note that this relative position between vehicle 1 and emergency vehicle 4 is illustrated in the figure 1 is by no means exhaustive.

[0028] A priority vehicle is characterized by five or six flashing lights, also called rotating beacons, emitting blue, red, or orange light. These colors are used by priority vehicles in all countries. Priority vehicles may have one or more rotating beacons, the arrangement of which can vary depending on the number of beacons they are equipped with (either a single beacon, a pair of separate beacons spaced apart, or several beacons aligned and close together), and their location on the vehicle's body (on the roof, on the front bumper, etc.). Rotating beacons also come in several types. They can be LED or incandescent bulbs.

[0029] The flashing lights of priority vehicles are also defined by their flashing nature, alternating between lit and unlit phases.

[0030] The process according to the invention will now be described with reference to figures 2 to 8 .

[0031] The method according to the invention comprises a step 100 of acquiring a sequence of images, including, for example, a first image I1, a second image I2, subsequent to the first image 11, and a third image I3 subsequent to the second image I2. Such images I1, I2, and I3 are represented in the figures 5a, 5b And 5c , respectively.

[0032] The process according to the invention includes a step 200 of colorimetric segmentation by thresholding.

[0033] This segmentation step 200 uses predefined segmentation thresholds to detect and segment bright areas ZL i in each image of the image selection. This colorimetric segmentation is performed according to four color categories: the colors red, orange, blue, and purple.

[0034] The color purple is notably used to detect certain specific flashing lights with a different chromaticity. For example, flashing lights with bulbs, perceived as blue to the naked eye, may be perceived as purple by cameras.

[0035] To accommodate the significant variability in the flashing lights of emergency vehicles, the thresholds used in the segmentation step 200 are broader than those typically used for traffic light recognition, for example. These segmentation thresholds are predefined for each color, for saturation, for light intensity, and for chrominance.

[0036] The 200 segmentation step gives the position in the image, the intensity and the color of the segmented light areas ZL i of the emergency vehicle, when these are lit.

[0037] As depicted in the figure 5a , at the end of the 200 colorimetric segmentation step, a plurality of colored luminous areas ZL 1, ZL 2, ZL 3, ZL 4, ZL 5 and ZL 6 are detected in the first image I1, likely to be flashing lights of priority vehicles.

[0038] Extending the threshold values ​​used for the segmentation step allows for adaptation to the variability of emergency vehicle beacons, enabling sensitivity to a wider range of color shades. However, this extension introduces significant noise during segmentation step 200.

[0039] To reduce the number of potential candidates for detecting emergency vehicle flashing lights, in other words, to reduce the number of false positives, the method according to the invention includes a post-segmentation step 210. This post-segmentation step 210 enables filtering based on predetermined criteria of the position of the luminous areas ZL i in the image under consideration, and / or the size of the luminous areas ZL i and / or the color of the luminous areas ZL i and / or the intensity of the luminous areas ZL i.

[0040] With reference to the figure 3The post-segmentation step 210 includes a dimensional filtering substep 211 in which bright areas ZL i with a size smaller than a predetermined dimensional threshold, for example, less than 20 pixels (referred to as small areas), are filtered out. This dimensional filtering is performed, in particular, in portions of the image far from a horizon line H, which represents infinity, and from a vanishing point F, namely at the edges of the image. In other words, this filter removes small bright areas ZL i that are present in a portion of the image where it is not common to encounter such areas. Indeed, when small bright areas ZL i are present in an image, they correspond to lights that are far away from vehicle 1 when these bright areas ZL i are close to the horizon line H.Thus, a small luminous area ZL i that is not close to the horizon line H would not correspond to ambient light distant from vehicle 1 but to measurement noise, hence the advantage of filtering such a luminous area ZL i. This is therefore the case for the luminous area ZL 1 illustrated in Figure 1. figure 5a .

[0041] Furthermore, as mentioned previously, the bright areas corresponding to distant lights are located close to the horizon line H and the vanishing point F of image I1. Consequently, bright areas located too far from this horizon line H and vanishing point F, particularly on the lateral edges of image I1, are also filtered out when they are smaller than the predetermined dimensional threshold. This is therefore the case for the bright area ZL 4 illustrated in the... figure 5a .

[0042] Post-segmentation step 210 includes substep 212, which filters out light areas with an intensity below a predetermined threshold, for example, less than 1000 lux, when these light areas (ZL i) exceed a predetermined size, for example, greater than 40 pixels. This is because the light areas filtered in substep 212, while having a size in the image corresponding to the proximity of the light to vehicle 1 in the rear scene filmed by camera 2, do not have the necessary light intensity to be considered suitable candidates for emergency vehicle beacons. A low-intensity segmented light area (ZL i) that is close enough to vehicle 1 to have a significant size in the image could, for example, simply be a reflection of sunlight on a surface.

[0043] Post-segmentation step 210 includes a further substep 213 of positional filtering in which bright areas positioned below the horizon line H are filtered. This is therefore the case for bright areas ZL 2 and ZL 3 illustrated in the figure 5a Indeed, such a position (below the horizon line H) of the light areas is characteristic of the front headlights of following vehicles in particular, and not of the flashing lights of priority vehicles which are positioned above the horizon line H.

[0044] Post-segmentation step 210 may further include a substep 214 for filtering conflicting light areas. At least two light areas are in conflict when they are close together, intersect, or if one is contained within the other. When such a conflict is detected, only one of these two light areas is retained, the other being filtered out. The light area among the two conflicting areas to be removed is determined, in a manner known per se, according to predetermined criteria for the brightness, size, and color of said area.

[0045] Post-segmentation step 210 includes, for a segmented light area, a sub-step 215 of oriented chromatic thresholding filtering.

[0046] With reference to the figure 8A color space (U,V) ranging from 0 to 255 is illustrated. Each color, R for red, G for violet, B for blue, and O for orange, is defined in this color space according to threshold values ​​Umax, Umin, Vmax, Vmin. For example, for the color blue, Umax(B), Umin(B), Vmax(B), Vmin(B).

[0047] Defining colors using minimum and maximum values ​​on the U and V axes creates rectangular blocks of color that are not representative of reality and therefore unsuitable. It is therefore necessary to perform chromatic filtering for each color. Oriented chromatic thresholding involves adjusting the minimum and maximum values ​​on the U and V axes of the color space for each color block. This reduces the risk of false detections due to the proximity of color blocks (proximity of U and V values ​​between colors in the (U,V) color space). For example, this oriented chromatic filter aims to filter out blue (B) colors that are too close to violet (V), red (R) colors that are too close to orange (O), and so on.

[0048] This oriented chromatic filtering allows for a refinement of color definition and, consequently, filters out a large number of false positives in the detection of bright areas.

[0049] Thus, after the implementation of the aforementioned sub-steps of post-segmentation step 210, only the following light zones (ZL i) are considered as potential candidates for emergency vehicle beacons: of a sufficiently large size (greater than 20 pixels); positioned above the horizon line H; sufficiently bright (light intensity greater than 1000 lux); and whose chromaticity is assured (with U and V values ​​adapted to avoid false positives).

[0050] It should be noted that not all sub-steps are necessarily always implemented to carry out the process according to the invention, but that only some of them, alone or in combination, could be retained, on a case-by-case basis, depending on the complexity of the image being processed.

[0051] Step 300 is a tracking step for each detected light zone ZLi in each image. As is known to those skilled in the art, an expected position for the segmented light zones ZLi in the image during segmentation step 200 is calculated by the computer 3 and used to ensure that a light detected in an image In corresponds to the same segmented light zone in a previous image In-1 that has moved. The expected position of the light zones is determined using a prediction. The expected position of the segmented light zone in the current image In is calculated from the position of the light zone in the previous image In-1, to which is added a vector corresponding to the movement of the light zone between image In-2 and image In-1, taking into account the movement of vehicle 1.

[0052] Furthermore, as explained below, the emergency lights of priority vehicles are characterized by their flashing frequency. To estimate this frequency, it is necessary to estimate the evolution of their brightness (alternating periods of illumination and inactivity) over time. Step 200, the segmentation process, provides the position, intensity, and color of these segmented light zones (ZL i) within the image only when they correspond to periods when the emergency lights are illuminated. Step 300, the tracking process, allows us to associate the emergency lights from one image to another and to extrapolate their positions when they are in an inactive phase (off).

[0053] This 300 tracking step, known from the prior art, has thresholds adapted to the flashing nature of the gyro beacons and allows in particular to associate the luminous areas of the gyro beacons from one image to another image, as well as to extrapolate their positions when the gyro beacons are in an off phase (corresponding to an absence of a corresponding luminous area in the image).

[0054] Each luminous zone ZL i segmented during the segmentation step 200 is associated, in a way known per se, with a luminous prediction zone ZP i of the same color.

[0055] For each segmented light zone ZL i for which no association was found with a prediction light zone Zpi during the follow-up step 300, a second segmentation step 310 is implemented, at the end of the follow-up step 300.

[0056] This second segmentation step 310 includes a first substep 311 in which the segmentation thresholds are broadened (in other words, the segmentation thresholds are defined to be less strict, less filtering), and the segmentation steps 200 and 300 of tracking are repeated for each image in the image sequence, with these new, broadened segmentation thresholds. This step enables the detection of a segmented bright area ZL i within a predicted segmentation area Zpi.

[0057] If, at the end of this first substep 311, no association is still found between the processed segmented light area ZL i and a predicted light area ZP i, a second substep 312 is implemented in which the segmentation thresholds are modified to correspond to the color white. This final check ensures that the false detection was indeed a false detection and not a headlight from a following vehicle.

[0058] This second sub-step 312 makes it possible in particular to detect headlights of following vehicles whose white light may contain the color blue for example.

[0059] The process according to the invention then includes a step 400 of colorimetric classification of each luminous zone ZL i.

[0060] This classification step 400 allows the selection of the light zones ZL i from the segmentation step 200. For each of the colors a classifier is trained (in a prior training step called offline) to discriminate positive data (representative of flashing lights to be detected) from negative data (representative of all the noises from the segmentation step 200 that are not flashing lights and that we therefore do not want to detect, such as front or rear lights of vehicles, reflections of the sun, signal lights, etc).

[0061] If during the classification step a segmented luminous area ZL i cannot be classified (recognized) by the classifier, then this luminous area is filtered.

[0062] Conversely, if a luminous zone ZL i is recognized by the classifier, it is considered a strong candidate to be a rotating beacon. Following classification step 400, a list of candidate luminous zones ZC i is obtained, these candidate luminous zones ZC i being characterized by the following parameters: a blinking state; a classification confidence index I cc; a position in the image; a color.

[0063] The flashing status is obtained by detecting the flashing of the rotating beacons. This detection consists of: a count of the number of images where the flashing light is on and, consequently, where the corresponding light zones ZL 5 and ZL 6 are detected (image I1 with reference to the figure 5a ), a count of the number of images where the flashing light is off and, consequently, where the corresponding light zones ZL 5 and ZL 6 are not detected (image I2 in reference to the figure 5b ), and a count of the number of images where the flashing light is switched on again and, consequently, where the corresponding light zones ZL 5 and ZL 6 are detected again (image I3 referring to the figure 5c ).

[0064] The confidence index I cc is obtained with the blinking information (blinking state) and positive classification by the classifier at step 400. According to one embodiment, after the classification step 400, the confidence index of each segmented light zone ZL i is updated.

[0065] If the classification is positive, the update of the classification confidence index Icc for an image at time t is done with respect to a classification confidence index Icc for an image at time t-1 according to the following formula: Icc t = Icc t − 1 + FA with FA a predetermined growth factor.

[0066] If the classification is negative, the update of the classification confidence index Icc for an image at time t is done with respect to a classification confidence index Icc for an image at time t-1 according to the following formula: Icc t = Icc t − 1 − FR with FR a predetermined reduction factor.

[0067] Position and color information, on the other hand, is given by classification step 200.

[0068] Following classification step 400, in order to determine whether a candidate luminous area ZC i is likely to be an emergency vehicle beacon, the method includes a frequency analysis step to calculate and threshold the flashing frequency of the segmented luminous area ZL i and a time integration calculation step of the classifier's response. These steps are detailed below.

[0069] In a step 500, a frequency analysis of each segmented luminous zone ZL i makes it possible to determine whether the segmented luminous zone ZL i is flashing or not.

[0070] Advantageously, prior to step 500, inconsistencies in the segmented bright areas ZL i are corrected. Specifically, this correction affects the color, size, and intensity of the segmented bright areas ZL i. If excessive color fluctuations are detected—for example, if the segmented bright area ZL i changes from red to orange between images—then said segmented bright area ZL i is filtered. Similarly, if the size of the segmented bright area ZL i varies too much from one image to another (a variation greater than two, for example), then said segmented bright area ZL i is filtered.

[0071] From the detection of the on and off phases of the rotating beacon, allowing the flashing to be determined, a fast Fourier transform (FFT) in a manner known in itself allows the frequency of this flashing to be determined.

[0072] During the frequency analysis step 500, the flashing frequency of each segmented light zone ZL i is compared to a predetermined first frequency threshold SF 1 and a second frequency threshold SF 2, which is higher than the first SF 1 threshold. If the flashing frequency is lower than the first SF 1 threshold, then the segmented light zone ZL i is considered non-flashing and therefore not a rotating beacon, and is filtered out. If the flashing frequency is higher than the second SF 2 threshold, then the segmented light zone ZL i is also considered not a rotating beacon and is filtered out.

[0073] This frequency analysis of the flashing pattern allows us to filter out segmented light zones (ZL i) that are known to be constant, flashing too slowly, or, conversely, flashing too rapidly to be emergency vehicle beacons. Thus, only segmented light zones (ZL i) with a flashing frequency between the frequency thresholds SF 1 and SF 2 are retained as suitable candidates.

[0074] In one embodiment, the first frequency threshold SF1 is equal to 1 Hz and the second frequency threshold SF2 is equal to 5 Hz. Furthermore, in another example, the flashing lights on police and emergency vehicles typically have a frequency between 60 and 240 FPM. FPM (short for flashes per minute) is a unit of measurement used to quantify the flashing frequency of a flashing light, corresponding to the number of cycles occurring in one minute. A value measured in FPM can be converted to hertz by dividing it by 60. Thus, for such emergency vehicles, their frequency is between 1 Hz and 6 Hz.

[0075] These frequency threshold values ​​SF1 and SF2 allow, in particular: to adapt to the camera (indeed, the afterimage of the eye does not allow us to see that some lights are flashing, as is the case with the lights on the roof of taxis, but the resolution of a camera can detect it); to obtain a more robust result against possible errors in the 200 segmentation step.

[0076] In an optional step 600, a directional analysis of each segmented light zone ZL i is performed to determine its displacement relative to vehicle 1 by tracking it in the sequence of images acquired by the camera. Thus, if a segmented light zone ZL i moves away from vehicle 1 (in other words, if the segmented light zone ZL i moves towards the vanishing point F or the horizon line H in the image), then the segmented light zone ZL i is filtered out. This could also be the case if the segmented light zone ZL i is stationary. This step makes it possible, in particular, to filter out segmented light zones ZL i that would indeed correspond to flashing lights of emergency vehicles moving in the opposite direction to that of vehicle 1, so that vehicle 1 does not have to consider these emergency vehicles.This step also allows the rear lights of cars travelling in a lane opposite to that of vehicle 1 to be filtered.

[0077] This step improves the detection performance (true positives and false positives) of emergency vehicle beacons by merging information relating to segmented light zones ZL i. Indeed, by studying the segmented light zones ZL i as a whole (in the entire image of the image sequence) and not individually, it is possible to reduce the false positive rate while maintaining a satisfactory detection rate.

[0078] At this stage of the process, a set of segmented luminous areas ZL i are detected in the image sequence. These detected segmented luminous areas ZL i are stored in a memory of the computer 3 as a list comprising an identifier associated with each segmented luminous area ZL i as well as parameters of these segmented luminous areas ZL i, such as: their position in the image; their size; their color; their intensity; their blinking state; the confidence index of the I CC classification associated with them.

[0079] At this stage, all detected and retained segmented luminous areas ZL i are considered potential emergency vehicle beacons. To determine the presence or absence of emergency vehicle beacons in the scene (the rear environment of the vehicle corresponding to the image sequence acquired by camera 2), the method includes a scene analysis step 700 consisting of calculating an overall confidence index I CG for each image in the image sequence, for each of the colors red, orange, blue, and violet. According to one embodiment, the overall confidence indices can be grouped by color. For example, the confidence index for the color violet is integrated into the confidence index for the color blue.

[0080] For this purpose, an instant confidence index I CI is calculated for each image in the image sequence and for each of the colours red, orange, blue and violet.

[0081] Instantaneous confidence is calculated from the parameters of the segmented bright areas ZL i of the current image in the image sequence. Segmented bright areas ZL i with a blinking state and a sufficient classification confidence index I CC are considered first.

[0082] To determine the segmented luminous areas ZL i exhibiting a classification confidence index I CC sufficient to be taken into account, a hysteresis state machine is illustrated in the figure 7 .

[0083] The state of the illuminated zone ZL i is initialized (Ei) to the "OFF" state: The transition from the "OFF" state to the "ON" state occurs if the classification confidence index ICC is greater than a predetermined threshold C3; the transition from the "ON" state to the "OFF" state occurs if the classification confidence index ICC is less than a predetermined threshold C4. An example of calculating this instantaneous confidence index ICI for an image and for a color could be the sum of the classification confidence indices ICC of all the segmented luminous areas ZL i detected in the image sequence, for that color. For example, for the color red: Ici Rouge = ∑ Icc Rouge

[0084] This instantaneous confidence is then filtered over time to obtain an overall confidence index for each image in the image sequence and for each of the colors red, orange, blue, and purple, according to the following formula: ICG t = 1 − α ∗ ICG t − 1 + α ∗ Ici Here, the instant confidence index of the color in question is;α being a predetermined coefficient, associated with a color, and allowing the weight of the instantaneous confidence index to be determined. Here in the calculation of the overall confidence index (ICG), for example, the value of the coefficient α is between 0 and 1, and preferably between 0.02 and 0.15.

[0085] The higher the value of the coefficient α, the more important the weight of the instantaneous confidence index Here is in the calculation of the overall confidence index I CG.

[0086] According to an example embodiment, the coefficient α varies according to the parameters of the segmented light zones ZL i. For example, the coefficient α is: increased when several segmented light zones ZL i have a classification confidence index I CC greater than predetermined threshold; decreased when computer 3 already indicates the presence of an emergency vehicle in the scene, which makes it possible to stabilize and robustify the result of the process in case of momentary loss of detection in the detection chain of segmented light zones ZLi, this momentary loss of detection being, for example, due to an occlusion, too much brightness or too much measurement noise.

[0087] According to one embodiment, the coefficient α varies depending on the position of the segmented luminous areas ZL i in the image. In particular, the coefficient α is: decreased if the image contains an isolated segmented bright area ZL i positioned below the horizon line H (in other words, the integration speed will be low); increased if a plurality of segmented bright areas ZL i are positioned above the horizon line H (which corresponds to a high integration speed).

[0088] According to one embodiment, the coefficient α varies depending on the relative position of the segmented luminous areas ZL i with respect to each other in the image. In particular, the coefficient α is increased if several segmented luminous areas ZL i are aligned on the same line L.

[0089] Variations in the parameter α allow adjustment of the sensitivity of the process and thus detection of events more or less quickly according to the parameters of the segmented luminous zones ZL i.

[0090] It is still possible to assign a weight to each segmented luminous area ZL i of the current image in the image sequence, which depends on other parameters, for example: The value of this weight is decreased if the segmented light area ZL i is small (i.e., less than 20 pixels, for example) and / or close to the horizon line H; the value of this weight is decreased if the segmented light area is dim (for example, with a light intensity less than 1000 lux); the value of this weight is decreased if the color of the light is not clear (for example, in the case of saturated white light); the value of this weight is increased if the segmented light area ZL i is close to other similar segmented light areas ZL i (in terms of size, position, intensity, color).

[0091] This weighting accelerates the increase in the overall confidence index (I CG) when multiple segmented light zones (ZL i) have highly correlated positions, brightness levels, and colors. Slowing the increase in the overall confidence index (I CG) when the lights are small and of lower intensity reduces the false positive rate, although this results in slower detection of distant emergency vehicles, which is acceptable.

[0092] Step 700 significantly reduces the false positive rate detected while maintaining satisfactory detection rates for emergency vehicle flashing lights.

[0093] At the end of step 700 it is possible for computer 3 to indicate the presence of an emergency vehicle in the rear environment of vehicle 1, corresponding to the sequence of images acquired by camera 2, and in particular to declare that a luminous area ZL i is an emergency vehicle beacon, for example using a hysteresis threshold known in itself.

[0094] A state machine in the form of hysteresis is illustrated in the figure 6 .

[0095] The state of the illuminated zone ZL i is initialized (Ei) to the "OFF" state: the transition from the "OFF" state to the "ON" state occurs if the overall confidence index ICG is greater than a predetermined threshold C1; the transition from the "ON" state to the "OFF" state occurs if the overall confidence index ICG is greater than a predetermined threshold C2.

[0096] The vehicle (in the case of an autonomous vehicle) or the driver of vehicle 1 otherwise, may then take the necessary measures to facilitate and not hinder the movement of said emergency vehicle.

Claims

1. Method for processing a video stream of images captured by at least one colour camera (2) on board a motor vehicle (1), said images being used by a computer (3) on board the vehicle to detect a priority vehicle (4) located in the vicinity of the vehicle (1), the at least one camera being directed towards the rear of the vehicle, the method being characterized in that it comprises the following steps: - a step (100) of acquiring a sequence of images; for each image of the sequence of images: - a step (200) of performing thresholding-based colourimetric segmentation, making it possible to detect coloured light areas (ZLi) of the images that are likely to be revolving lights (5, 6); - a step (300) of tracking each segmented light area (ZLi), according to which each light area segmented during the segmentation step (200) is associated with a prediction light area (ZPi) of the same colour; - a step (400) of colour classification, by a previously trained classifier, of each segmented light area (ZLi); - a step (500) of frequency analysis of each segmented light area (ZLi) to determine a flashing characteristic of said segmented area; - a step (700) of calculating an overall confidence index (ICG) for each image in the image sequence, taking into account a classification confidence index obtained from the colour classification step (400), which allows a segmented light area (ZLi) to be declared as a revolving light.

2. Method as claimed in claim 1, characterized in that, in the segmentation step (200), predefined segmentation thresholds are used so as to segment the light areas (ZLi) according to four categories: - the colour red, - the colour orange, - the colour blue, and - the colour violet.

3. Method according to any of claims 1 or 2, characterized in that, after the segmentation step (200), the method further comprises a post-segmentation filtering step (210) for filtering the results obtained from the segmentation step (200), this post-segmentation stage being carried out according to the predetermined criteria of position, and / or size, and / or colour, and / or intensity.

4. Method according to claim 3, characterized in that the post-segmentation step (210) comprises a dimensional filtering sub-step (211) wherein the bright areas (ZLi) located in parts of the image far from a horizon line (H) and from a vanishing point (F) and having a size smaller than a predetermined dimensional threshold are filtered out.

5. Method as claimed in claim 3, characterized in that the post-segmentation step (210) comprises a sub-step (212) of filtering light areas (ZLi) having a size greater than a predetermined dimensional threshold and a light intensity less than a predetermined light intensity threshold.

6. Method as claimed in claim 3, characterized in that the post-segmentation step (210) comprises a positional filtering sub-step (213) in which light areas (ZLi) positioned below a horizon line (H) defined on the image of the sequence of images are filtered.

7. Method as claimed in claim 3, characterized in that the post-segmentation step (210) comprises, for a segmented light area (ZLi), a sub-step (215) of performing oriented chromatic thresholding-based filtering.

8. Method as claimed in any one of the preceding claims, characterized in that it furthermore comprises a second segmentation step (310), at the end of the tracking step (300), for each segmented light area (ZLi) for which no association was found.

9. Method as claimed in claim 8, characterized in that the second segmentation step (310) comprises: - a first sub-step (311) in which the segmentation thresholds are widened, and the segmentation (200) and tracking (300) steps are repeated for each image of the sequence of images with these new widened segmentation thresholds, the segmentation thresholds being those corresponding to the colour of the segmented light area, and - if, at the end of this first sub-step, no association has been found, a second sub-step (312), in which the segmentation thresholds are modified so as to correspond to those of the colour white.

10. Method according to any one of the foregoing claims, characterized in that, during the frequency analysis step (500), a flashing frequency of each segmented light area (ZLi) is compared with a first frequency threshold (SF1) and with a second frequency threshold (SF2) greater than the first frequency threshold (SF1), both thresholds being predetermined, a segmented light area (ZLi) being filtered if: - its flashing frequency is less than the first frequency threshold (SF1), such that the segmented light area is considered not to be flashing or weakly flashing, and therefore not to be a revolving light; - its flashing frequency is greater than the second frequency threshold (SF2), such that the light area is also considered not to be a revolving light.

11. Method as claimed in claim 10, characterized in that the first frequency threshold (SF1) is equal to 1 Hz and the second frequency threshold (SF2) is equal to 5 Hz.

12. Method according to any of claims 1 to 11, characterized in that it further comprises a step (600) of directional analysis of each segmented light area (ZLi) to determine a movement of said segmented light area.

13. Method as claimed in the preceding claim, characterized in that a segmented light area (ZLi) is filtered if the displacement direction obtained in the directional analysis step (600) makes it possible to conclude as to: - immobilization of the segmented light area (ZLi), with respect to the vehicle (1); - moving away of the segmented light area (ZLi), with respect to the vehicle (1).

14. A computer program product, comprising instructions for implementing a method comprising: - a step (100) of acquiring a sequence of images; for each image of the sequence of images: - a step (200) of performing thresholding-based colourimetric segmentation, making it possible to detect coloured light areas (ZLi) of the image that are likely to be rotating lights (5, 6); - a step (300) of tracking each segmented light area (ZLi), according to which each segmented light area (ZLi) in the segmentation step (200) is associated with a prediction light area (ZPi) of the same colour; - a step (400) of performing colourimetric classification, using a previously trained classifier, of each segmented light area (ZLi); - a step (500) of performing frequency analysis of each segmented light area (ZLi), making it possible to determine a flashing nature of the segmented light area; - a step (700) of calculating an overall confidence index (ICG) for each image in the image sequence, taking into account a classification confidence index obtained from the colour classification step, enabling a light area (ZLi) to be declared as a revolving light, when implemented by a computer.

15. Vehicle (1), comprising at least one colour camera (2) oriented towards the rear of the vehicle and capable of acquiring a video stream of images of the rear environment of the vehicle, and at least one computer (3), the computer (3) being configured to implement: - a step (100) of acquiring a sequence of images; for each image of the sequence of images: - a colour segmentation step (200) using thresholding to detect coloured light areas (ZLi) in the image that are likely to be revolving lights (5, 6); - a step (300) of tracking each segmented light area (ZLi), according to which each light area (ZLi) segmented in the segmentation step (200) is associated with a prediction light area (ZPi) of the same colour; - a step (400) of performing colourimetric classification, by a previously trained classifier, of each segmented light area (ZLi); - a step (500) of performing frequency analysis of each segmented light area (ZLi), making it possible to determine a flashing nature of the segmented light area; - a step (700) of calculating an overall confidence index (ICG) for each image in the image sequence, taking into account a classification confidence index obtained from the colour classification step, enabling a segmented luminous area (ZLi) to be declared as a revolving light.