Image Processing Method

Abstract

The present invention relates to a method for processing a video stream of images taken by a color camera (2) for use by a computer (3) on board a motor vehicle (1) to detect priority vehicles (4), the method comprising: an acquisition step (100) for acquiring an image sequence; For each image in the image sequence, a segmentation step (200) in which a colorimetric segmentation based on thresholding is performed in order to be able to detect colored luminous regions; a tracking step (300) of tracking each segmented luminous region; a classification step (400) for performing a colorimetric classification of each segmented luminous region; a frequency analysis step (500) in which a frequency analysis is carried out on each segmented luminous area in order to be able to determine the flashing nature of said area; a calculation step (700) for calculating an overall confidence index (ICG) for each image of the image sequence in order to be able to declare a segmented light area as a flashing light; A method is provided, comprising:

Description

[Technical Field]

[0001] Technical Field The present invention relates to an image processing method, and more particularly to an image processing method for detecting emergency vehicles.

[0002] Conventional technology Today, it is known to equip motor vehicles with driver assistance systems, commonly referred to as ADAS ("Advanced Driver Assistance Systems"). As is known, such systems comprise an imaging device, such as a camera, mounted on the vehicle and capable of generating a series of images representative of the vehicle's environment. For example, a camera mounted on the rear of the vehicle can capture images of the environment behind the vehicle, in particular of following vehicles. These images are used by a processing unit to assist the driver, for example by detecting obstacles (pedestrians, stopped vehicles, objects on the road, etc.) or by estimating the time until a collision with an obstacle. Therefore, the information provided by the images acquired by the camera must be sufficiently reliable and relevant to enable the system to assist the driver of the vehicle.

[0003] In particular, many international legal authorities stipulate that ordinary drivers must not obstruct the passage of priority vehicles (also known as emergency vehicles, such as fire engines, ambulances, and police vehicles) in response to a traffic emergency and must allow such priority vehicles to pass easily. Therefore, it is appropriate for ADAS systems to be able to recognize such priority vehicles so as not to obstruct their dispatch, and even better, to be able to do so when their lights (flashing lights) are activated.

[0004] In current ADAS systems equipped with cameras capturing images of the front or rear of the vehicle, priority vehicles are detected in the same way as other standard (non-priority) vehicles. These systems typically implement a combination of machine learning and geometric perception approaches. Images from these cameras are processed to extract bounding boxes around all types of vehicles (cars, trucks, buses, motorcycles, etc.), including emergency vehicles. This means that existing ADAS systems cannot reliably distinguish between a following priority vehicle and a following standard vehicle.

[0005] Furthermore, such systems suffer from problems with partial or total temporal obscuration of the images acquired by the camera, which is related to the fact that priority vehicles do not necessarily follow normal traffic rules and are allowed to zigzag between lanes, reduce safety distances or even drive across two lanes, which means that existing systems are not adapted to such behavior and situations.

[0006] Additionally, detecting priority or emergency vehicles is challenging due to the wide variety of priority vehicles. In practice, these vehicles feature flashing lights that can be LED-based or tube-based, fixed or rotating, various colors, and variably positioned on the vehicle. For example, some vehicles are equipped with a single flashing light, others with a pair of flashing lights, and still others with a bar containing three or more flashing lights. This variation makes it even more difficult for existing ADAS systems to reliably detect priority vehicles.

[0007] These are exacerbated by the problem of detecting scenes that include the front headlights and taillights of other vehicles and all other lights in the environment behind the vehicle, which can increase the difficulty in detecting the flashing lights of priority vehicles.

[0008] Presentation of the invention Therefore, according to the present invention, an image processing method is proposed for quickly and reliably detecting priority vehicles, regardless of the type of priority vehicle and its running state, in particular by detecting the lights (flashing lights) of these vehicles.

[0009] According to the present invention, the above problem is solved by a method for processing a video stream of images taken by at least one color camera mounted on a motor vehicle, which is used by a computer mounted on said vehicle to detect priority vehicles located in the environment of the vehicle, the at least one camera being directed towards the rear of the vehicle, said method comprising: an acquisition step for acquiring an image sequence; For each image in the image sequence, a segmentation step in which a colorimetric segmentation based on thresholding is performed in order to be able to detect colored luminous areas of the image that are likely to be flashing lights; a tracking step of tracking each segmented luminous region and associating each segmented luminous region in the segmentation step with a predicted luminous region of the same color according to the tracking; a classification step, performing a colorimetric classification of each segmented luminous region using a previously trained classifier; a frequency analysis step of performing a frequency analysis of each segmented luminous area to determine the flashing characteristics of the segmented luminous area; A computation step for computing an overall confidence index for each image in the image sequence in order to be able to declare a segmentation light area as a flashing light; The problem is solved by a method including:

[0010] The method according to the invention therefore makes it possible to reliably detect the flashing lights of emergency vehicles at distances of up to 150 metres, regardless of lighting and weather conditions.

[0011] According to one exemplary embodiment, in the segmentation step, four categories are identified: ·red, ·Orange, Blue, and ·violet, A predetermined segmentation threshold is used to segment the luminous regions according to

[0012] According to one embodiment, after the segmentation step, the method further comprises a so-called post-segmentation filtering step that allows filtering out the results from the segmentation step, said post-segmentation step being performed according to predetermined criteria regarding position and / or size and / or color and / or intensity. Such filtering makes it possible to reduce false positives.

[0013] According to one exemplary embodiment, the post-segmentation step comprises a dimension filtering sub-step of filtering out luminous regions located in image parts away from the horizon and away from the vanishing point and having a size below a predetermined dimension threshold, which makes it possible to eliminate candidates corresponding to objects perceived by the camera that correspond to measurement noise.

[0014] According to one exemplary embodiment, the post-segmentation step includes a sub-step of filtering out luminous regions having a size above a predetermined dimension threshold and an luminous intensity below a predetermined luminous intensity threshold, thereby eliminating candidates that are close to the vehicle but do not have the required luminous intensity for a flashing light.

[0015] According to one exemplary embodiment, the post-segmentation step comprises a position filtering sub-step of filtering out luminous areas located below a horizon line defined on the images of the image sequence, thereby eliminating candidates corresponding to the headlights of a following vehicle.

[0016] According to one embodiment, the post-segmentation step includes a sub-step of filtering the segmented luminous regions based on directional color thresholding, where the specific filtering allows for more accurate filtering of colors, for example, for the color blue, filtering out many of the false positives in the detection of luminous regions classified as blue due to the fact that white light emitted from the headlights of a following vehicle may be perceived as blue by the camera.

[0017] According to one embodiment, the method further comprises a second segmentation step for each segmented luminous area for which no relevance was found at the end of the tracking step.

[0018] According to an exemplary embodiment, the second segmentation step comprises: a first substep of increasing the segmentation threshold and repeating the segmentation and tracking steps for each image of the image sequence with the newly increased segmentation threshold corresponding to the color of the segmented luminous region; a second substep in which, if no associations are found at the end of the first substep, the segmentation threshold is modified to correspond to the white segmentation threshold; Includes:

[0019] This step allows the segmentation to check the segmented (detected) luminous regions, and indeed this final check ensures that false positives are truly false positives and not, for example, the headlights of a following vehicle.

[0020] According to an exemplary embodiment, in the frequency analysis step, the flashing frequency of each segmentation light-emitting region is compared with a first frequency threshold and a second frequency threshold greater than the first frequency threshold, where both thresholds are predetermined; If the flashing frequency is less than a first frequency threshold, the segmentation light-emitting area is not flashing or is only flashing weakly, and is therefore not considered to be a flashing light; If the flashing frequency is greater than the second frequency threshold, the segmentation light-emitting area is also not considered to be a flashing light. Then, the segmentation luminous regions are filtered out.

[0021] According to one exemplary embodiment, the first frequency threshold is equal to 1 Hz and the second frequency threshold is equal to 5 Hz.

[0022] According to an exemplary embodiment, the method further comprises a directional analysis step, in which a directional analysis of each segmentation light-emitting area is performed in order to be able to determine the displacement of said segmentation light-emitting area.

[0023] According to an exemplary embodiment, the displacement direction obtained in the direction analysis step is - The segmentation light emitting area is fixed relative to the vehicle; The segmentation light-emitting area is moving away from the vehicle. If it can be concluded that the segmentation is complete, the luminous regions are filtered out.

[0024] The present invention also provides a computer program product which, when executed by a computer, an acquisition step for acquiring an image sequence; For each image in the image sequence, a segmentation step in which a colorimetric segmentation based on thresholding is performed in order to be able to detect colored luminous areas of the image that are likely to be flashing lights; a tracking step of tracking each segmented luminous region and associating each segmented luminous region in the segmentation step with a predicted luminous region of the same color according to the tracking; a classification step, performing a colorimetric classification of each segmented luminous region using a previously trained classifier; a frequency analysis step of performing a frequency analysis of each segmentation luminous region in order to be able to determine the flashing property of the segmentation luminous region; A computation step for computing an overall confidence index for each image in the image sequence in order to be able to declare a segmentation light area as a flashing light; The present invention relates to a computer program product comprising instructions for implementing a method comprising:

[0025] The present invention also provides a vehicle comprising at least one color camera directed toward the rear of the vehicle and capable of acquiring a video stream of images of the environment behind the vehicle, and at least one computer, the computer being configured to: an acquisition step of acquiring a plurality of images; For each image in the image sequence, a segmentation step in which a colorimetric segmentation based on thresholding is performed in order to be able to detect colored luminous areas of the image that are likely to be flashing lights; a tracking step of tracking each segmented luminous region and associating each segmented luminous region in the segmentation step with a predicted luminous region of the same color according to the tracking; a classification step, performing a colorimetric classification of each segmented luminous region using a previously trained classifier; a frequency analysis step of performing a frequency analysis of each segmentation luminous region in order to be able to determine the flashing property of the segmentation luminous region; A computation step for computing an overall confidence index for each image in the image sequence in order to be able to declare a segmentation light area as a flashing light; The present invention relates to a vehicle configured to achieve the above.

[0026] Other features, details and advantages will become apparent from reading the following detailed description and examining the accompanying drawings. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a schematic diagram showing a vehicle and a priority vehicle according to the present invention; [Figure 2] FIG. 1 illustrates an exemplary implementation of the method according to the invention. [Figure 3] FIG. 2 illustrates an exemplary embodiment of the post-segmentation step of the method according to the invention. [Figure 4] FIG. 3 shows an exemplary embodiment of the second segmentation step of the method according to the invention. [Figure 5a] FIG. 1 shows a first image of an image sequence processed by the method according to the invention; [Figure 5b] FIG. 2 shows a second image of an image sequence processed by the method according to the invention. [Figure 5c] FIG. 3 shows a third image of the image sequence processed by the method according to the invention. [Figure 6] FIG. 1 illustrates a state machine in the form of hysteresis. [Figure 7] FIG. 10 illustrates another state machine in the form of hysteresis. [Figure 8] FIG. 1 is a diagram illustrating a color space (U, V).

[0028] Description of the embodiment Figure 1 shows a schematic representation of a vehicle 1 equipped with a color camera 2 facing rearward of the vehicle 1 and capable of acquiring images of the environment behind the vehicle 1, and at least one computer 3 configured to use the images acquired by the camera 2. In this Figure 1, an emergency or priority vehicle 4 is located behind the vehicle 1, within the field of view of the camera 2. It should be noted that this relative position between the vehicle 1 and the emergency vehicle 4 shown in Figure 1 is in no way limiting.

[0029] Priority vehicles feature light spots 5, 6, also called flashing lights, that emit blue, red, or orange light. These colors are used by priority vehicles in all countries. Priority vehicles can include one or more flashing lights, and their placement can vary depending on the number of flashing lights installed (either a single flashing light, a pair of separate flashing lights spaced apart from each other, or multiple flashing lights closely aligned with each other) and the location of these flashing lights on the body of the priority vehicle (priority vehicle roof, priority vehicle front bumper, etc.). There are also many types of flashing lights. Flashing lights can be LED-based or bulb-based.

[0030] A priority vehicle's flashing light is also defined by its flashing nature, which alternates between an on phase and an off phase.

[0031] The method according to the invention will now be described with reference to FIGS.

[0032] The method according to the invention comprises a step 100 of acquiring a sequence of images, which may comprise, for example, a first image I1, a second image I2 following the first image I1, and a third image I3 following the second image I2. Such images I1, I2 and I3 are shown in Figures 5a, 5b and 5c, respectively.

[0033] The method according to the invention includes a segmentation step 200, which performs a colorimetric segmentation based on thresholding.

[0034] Such a segmentation step 200 allows the luminous regions Z 1 in each image of the selection of images to be identified using a predetermined segmentation threshold. L Such colorimetric segmentation can be performed to detect and segment four color categories: ·red, ·Orange, Blue, and ·violet, This is carried out in accordance with the following:

[0035] Purple is particularly used to detect certain flashing lights that have different chromaticities, for example, a tube-based flashing light may be perceived as blue by the naked eye but as purple by a camera.

[0036] To accommodate the significant variations in the flashing lights of priority vehicles, the thresholds used in the segmentation step 200 are expanded relative to thresholds used conventionally for traffic light recognition, for example, where the segmentation thresholds are predefined for each color, saturation, light intensity and chromaticity.

[0037] The segmentation step 200 segments the emergency vehicle's light-emitting area Z L When i is on, the light-emitting area Z L The position, intensity and color of i in the image are given.

[0038] As shown in FIG. 5a, at the end of the colorimetric segmentation step 200, the colored light-emitting region Z L 1,Z L 2,Z L 3,Z L 4,Z L 5 and Z L6 were detected, and these were likely to be the flashing lights of priority vehicles.

[0039] Increasing the threshold used in the segmentation step allows for adaptation to the variability of priority vehicle flashing lights to gain sensitivity to a wider range of hues, however, such an increase introduces more noise into the segmentation step 200.

[0040] In order to reduce the number of potential candidates for the detection of the flashing lights of the priority vehicle, in other words to reduce the number of false positives, the method according to the invention comprises a post-segmentation step 210, by which the luminous regions Z L position i and / or this luminous region Z L the size of i and / or this luminous area Z L color of i and / or this luminous area Z L Filtering can be performed based on predetermined criteria regarding the intensity of i.

[0041] Referring to FIG. 3, the post-segmentation step 210 identifies luminous regions Z 1 having a size below a predetermined dimension threshold, for example, a size below 20 pixels (referred to as small size). L i. Such dimensional filtering is performed in particular in image parts away from the horizon H, which represents infinity, and the vanishing point F, especially at the edges of the image. In other words, this filtering eliminates the small sized luminous areas Z L A small luminous region Z exists in an image area where i would not normally exist. L In practice, the small-sized light-emitting region Z L If i exists in the image, this luminous region Z L If the light source Z i is near the horizon H, it corresponds to light that is far away from the vehicle 1. Therefore, the small-sized light-emitting area Z i is not considered to be near the horizon H. Li corresponds to measurement noise rather than to ambient light far from the vehicle 1, and therefore, L It is beneficial to filter out the luminous area Z i shown in FIG. L This corresponds to case 1.

[0042] As mentioned above, a light-emitting area corresponding to distant light is located near the horizon H and the vanishing point F of the image I1. Here, if a light-emitting area located very far away from the horizon H and the vanishing point F, especially a light-emitting area at the edge of the image I1, has a size less than a predetermined threshold value, this light-emitting area is also filtered out. That is, this is the light-emitting area Z shown in FIG. 5a. L This is case 4.

[0043] The post-segmentation step 210 segments these luminous regions Z L If i has a size exceeding a predetermined threshold, for example a size of more than 40 pixels, the method includes a substep 212 of filtering out luminous regions having a luminous intensity below a predetermined luminous intensity threshold, for example below 1000 lx. In fact, the luminous regions filtered out in substep 212 have a size in the image corresponding to a proximity light relative to vehicle 1 in the rear scene photographed by camera 2, but do not have the luminous intensity required for a candidate of interest for the flashing lights of a priority vehicle. The segmentation luminous regions Z that are low in intensity but sufficiently close to vehicle 1 to have a so-called large size in the image are L i may correspond, for example, to the simple reflection of the sun's rays from the support.

[0044] The post-segmentation step 210 further includes a position filtering sub-step 213, which filters out luminous regions located below the horizon H. That is, this filters out the luminous region Z shown in FIG. L 2 and Z LThis is case 3. In fact, the position of the light-emitting area (a position below the horizon H) is characteristic of the front headlights of the following vehicle, not the flashing lights of the priority vehicle located above the horizon H.

[0045] The post-segmentation step 210 may also include a sub-step 214 of filtering out conflicting luminous regions. At least two luminous regions conflict when they are close to each other, when they intersect each other, or when they are contained within each other. If such a conflict is observed, only one of the two luminous regions is kept and the other is filtered out. The luminous region to be eliminated from the two conflicting regions is determined in a manner known per se according to predetermined criteria related to the brightness, size, and color of the region.

[0046] The post-segmentation step 210 includes a sub-step 215 of filtering the segmented luminous regions based on directional chromatic thresholding.

[0047] Referring to Figure 8, a color space (U, V) of 0 to 255 is shown. Each color, i.e., R representing red, V representing purple, B representing blue, and O representing orange, is calculated based on the threshold U max ,U min ,V max ,V min For example, in the case of blue, U max (B),U min (B), V max (B), V min (B).

[0048] Such color definitions based on minimum and maximum values ​​on the U and V axes do not represent reality and therefore form inappropriate rectangular color blocks. Therefore, color filtering must be performed for each color. Filtering based on directional color thresholding is equivalent to adjusting the minimum and maximum values ​​of the color block on the U and V axes of the color space for each color. This reduces the risk of false detection due to the proximity of color blocks (the proximity of the U and V values ​​of each color in the color space (U, V)). For example, the directional color filter is required to filter out blue B that is too close to purple V, red R that is too close to orange O, etc.

[0049] Such directional color filtering allows for refinement of color definitions, thus filtering out many false positives in the detection of luminescent regions.

[0050] Thus, after performing the above-described substeps of the post-segmentation step 210, all of the potential candidates for the priority vehicle flashing lights are retained as follows: · It has a sufficiently large size (more than 20 pixels); Located above the horizon H; · Sufficiently bright (with a luminous intensity of over 1000lx); Its chromaticity is guaranteed (with appropriate U and V values ​​to avoid false positives), Light-emitting area Z L It becomes i.

[0051] It should be noted that not all of the sub-steps necessarily need to be implemented to carry out the method according to the invention, and only some of the sub-steps can be retained on a case-by-case basis, either alone or in combination, depending on the complexity of the images to be processed.

[0052] Step 300 is performed by detecting each luminous region Z detected in each image. LIn a method known to those skilled in the art, in the segmentation step 200, the luminous region Z i is segmented in the image. L A predicted position of i is calculated by computer 3 and used to verify that the light detected in image In actually corresponds to the same segmented luminous region in the previous image In-1 and may have moved. Using the prediction, a predicted position of the luminous region is determined. Taking into account the displacement of vehicle 1, a predicted position of the segmented luminous region in the current image In is calculated based on the position of the luminous region in the previous image In-1 plus a vector corresponding to the displacement of the luminous region between images In-2 and In-1.

[0053] Furthermore, as will be explained below, the flashing lights of priority vehicles are characterized in particular by the flash frequency of said flashing lights. Now, in order to be able to estimate the flash frequency of a flashing light, it is necessary to be able to estimate the evolution of its brightness over a given time (alternating between lit and non-lit phases). In the segmentation step 200, these segmented luminous areas Z L Only when i corresponds to the on-phase of the flashing light, these segmentation luminous regions Z L Given the position, intensity and color of i in the image, the tracking step 300 allows for relating each flashing light from one image to another, and also allows for extrapolating the position of these flashing lights when they are in a non-illuminated (off) state.

[0054] Such a tracking step 300, known from the prior art, has a threshold adapted to the flashing nature of the flashing light and in particular makes it possible to associate the luminous areas of the flashing light from one image to another and also to extrapolate its position when the flashing light is in its off phase (a phase corresponding to the absence of a corresponding luminous area in the image).

[0055] Each luminous region Z segmented in the segmentation step 200 L i is a predicted light-emitting area Z of the same color in a manner known per se. P is associated with i.

[0056] Predicted luminous area Z at tracking step 300 P Each segmentation luminous region Z where no association with i was found L For i, a second segmentation step 310 is performed at the end of the tracking step 300 .

[0057] This second segmentation step 310 includes a first sub-step 311 of expanding the segmentation threshold (in other words, defining a looser segmentation threshold so that less filtering is required), and the segmentation step 200 and tracking step 300 are repeated for each image of the image sequence using the new expanded segmentation threshold, resulting in a predicted segmentation region Z P Segmentation of luminous region Z within i L i can be detected.

[0058] At the end of this first sub-step 311, the processed segmentation luminous area Z L i and predicted light emission area Z P If still no correlation is found between i and , a second substep 312 is performed in which the segmentation threshold is modified to correspond to the color white. In fact, this final check makes it possible to ensure that any false positives are truly false positives and are not the headlights of a following vehicle.

[0059] This second substep 312 makes it possible in particular to detect the headlights of following vehicles, whose light may for example comprise a blue colour.

[0060] The method according to the invention then comprises the steps of: L It includes a classification step 400, which performs a colorimetric classification of i.

[0061] The classification step 400 is performed by dividing the luminous region Z obtained from the segmentation step 200. L For each color, a classifier is trained (in the previous so-called offline training step) to discriminate positive data (representing flashing lights to be detected) from negative data (representing all the noise resulting from the segmentation step 200, e.g., vehicle headlights or taillights, sun reflections, traffic light lights, etc., that are not flashing lights and therefore not desired to be detected).

[0062] In the classification step, the segmentation luminous region Z L If i cannot be classified (recognized) by the classifier, this luminous region is filtered out.

[0063] On the other hand, the luminous region Z L If i is recognized by the classifier, it is retained as a strong candidate for a flashing light. At the end of the classification step 400, the candidate luminous region Z C A list of i is obtained, and these candidate luminous regions Z C i is each of the following parameters: ·Flushing state, ·Classification confidence index Icc, Position in the image, ·color, It is characterized by:

[0064] The flashing state is acquired by detecting the flashing of the flashing lamp. The flashing light is on and therefore the corresponding light-emitting area Z L 5 and Z L Counting the number of images in which 6 is detected (image I1 in FIG. 5a); The flashing light is off and therefore the corresponding light-emitting area Z L 5 and Z LCounting the number of images in which 6 is not detected (image I2 in FIG. 5b); The flashing light is turned on again and the corresponding lighting area Z L 5 and Z L Counting the number of images in which 6 is again detected (image I3 in FIG. 5c); It consists of:

[0065] The confidence index Icc is obtained in step 400 by the classifier using information about flashing (flashing state) and positive classification. According to one embodiment, after the classification step 400, each segmentation luminous region Z L The confidence index of i is updated.

[0066] If the classification is positive, the classification confidence index Icc of the image at time t is calculated relative to the classification confidence index Icc of the image at time t-1 using the formula: [Formula 1] Icc(t)=Icc(t-1)+FA where FA is a predetermined growth factor.

[0067] If the classification is negative, the classification confidence index Icc of the image at time t is calculated relative to the classification confidence index Icc of the image at time t-1 using the formula: [Formula 2] Icc(t) = Icc(t-1) - FR where FR is a predetermined reduction factor.

[0068] The position and color information is for the portion provided by the classification step 200 .

[0069] At the end of the classification step 400, the candidate luminous regions Z C To determine whether i is likely to be a flashing light of an emergency vehicle, the method includes segmenting the light-emitting region Z LThe method includes a frequency analysis step that performs a frequency analysis to calculate and threshold the flashing frequency of i, and a computation step that calculates the time integral of the classifier response, which are described in more detail below.

[0070] In step 500, each segmentation light emitting area Z L Segmentation of luminous region Z by performing frequency analysis of i L It is possible to determine whether i is a flushing or non-flushing state.

[0071] Advantageously, before this step 500, a segmentation light emitting area Z L In particular, the discrepancies that may occur in the segmentation luminous area Z L If excessive color variations are detected, e.g., segmentation of the luminous region Z L If i changes from red to orange for each image, the segmentation light-emitting area Z L i is filtered out. Another alternative is to segment the luminous area Z L If the change in size of i is excessively large (for example, if there is a change of more than 2), the segmentation light-emitting region Z L i is filtered out.

[0072] Since the flashing can be determined based on the detection of the on and off phases of the flashing lamp, it is possible to determine the frequency of said flashing by means of a Fast Fourier Transform (FFT), in a manner known per se.

[0073] In the frequency analysis step 500, each segmentation luminous region Z L The flashing frequency of i is equal to or exceeds a first frequency threshold S F 1 and is further compared with a first frequency threshold S F A second frequency threshold S greater than 1 F2, where both thresholds are predetermined. When the flashing frequency is greater than or equal to the first frequency threshold S F If it is less than 1, the segmentation luminous area Z L i is not flashing and is therefore considered not to be a flashing light and is filtered out. F If greater than 2, segmentation luminous area Z L i is still considered not to be a flashing light and is filtered out.

[0074] By analyzing the frequency of these flashes, the segmentation luminous area Z L i is constant or the light-emitting region Z L Segmentation of the luminous region Z where i is sure to be flashing slowly L i can be filtered out, or conversely, the light-emitting area Z L i is flashing at high speed and is definitely the flashing light of a priority vehicle. L i can be filtered out. Thus, all of the candidates retained as interesting candidates are filtered out to within the frequency threshold S F 1,S F Segmentation of the light-emitting area Z with flashing frequency between 2 L It becomes i.

[0075] According to one exemplary embodiment, a first frequency threshold S F 1 is equal to 1 Hz, and the second frequency threshold S F 2 is equal to 5 Hz. Furthermore, according to one embodiment, flashing lights installed on police and emergency vehicles typically have a frequency of 60 to 240 FPM. FPM (short for flashes per minute) is a unit of measurement used to quantify the flashing frequency of a flashing light and corresponds to the number of cycles occurring in one minute. Values ​​measured in FPM can be converted to Hertz by dividing by 60. In other words, the respective frequencies on such priority vehicles are between 1 Hz and 6 Hz.

[0076] These frequency thresholds S F 1 and S F 2, in particular, Camera adaptation (in fact, even if visibility is maintained, the flashing of certain lights, such as those mounted on the roof of a taxi, will not be observable, but the resolution of the camera will allow these lights to be detected), Obtaining more robust results in the face of any errors in the segmentation step 200; This makes it possible. In optional step 600, the segmentation light emitting area Z is calculated based on the vehicle 1. L Segmentation of the luminous region Z in the image sequence acquired by the camera to determine the displacement of i L By tracking i, each segmentation luminous region Z L A directional analysis of i is performed. Therefore, the segmentation luminous region Z L When i is moving away from vehicle 1 (in other words, the segmentation light emitting area Z L If i is close to the vanishing point F or the horizon H in the image, the segmentation luminous area Z L i is filtered out. This means that the segmentation light emitting area Z L This step may also be applicable when i is stationary. In particular, it is possible to obtain a plurality of segmented luminous areas Z of priority vehicles that correspond significantly to the flashing lights but are moving in the opposite direction to vehicle 1. L i can be filtered out, which means that vehicle 1 does not need to consider the priority vehicle. This step can also filter out the tail lights of vehicles traveling in the oncoming lane opposite to vehicle 1's lane.

[0077] This step generates a segmented light-emitting region Z LBy fusing information about i, it is possible to improve the detection performance (true positives and false positives) for detecting the flashing lights of emergency vehicles. In practice, the segmentation light-emitting area Z L By examining i as a whole (in all images of the image sequence) rather than individually, the false positive rate can be reduced while maintaining a satisfactory detection rate.

[0078] In this step of the method, the segmentation luminous area Z L A set of i is detected in the images of the image sequence. These detected segmentation luminous regions Z L i is the area of ​​each segmented luminous region Z L i and the identifier associated with these segmentation luminous regions Z L The i parameter, for example, their respective positions in the image, Each size, Each color, -Each strength, - Each flushing state, · The classification confidence index Icc associated with each The information is stored in the memory of the computer 3 in the form of a list including:

[0079] In this step, the detected and retained segmented luminous region Z L All of i are considered to be potential flashing lights of an emergency vehicle. To determine the presence or absence of flashing lights of an emergency vehicle in the scene (the environment behind the vehicle corresponding to the image sequence acquired by camera 2), the method finally includes a scene analysis step 700 consisting of calculating an overall confidence index ICG for each image of the image sequence for each of the colors red, amber, blue and purple. According to an exemplary embodiment, the overall confidence indexes may be grouped by color. For example, the confidence index for purple is combined with the confidence index for blue.

[0080] For this purpose, an instantaneous confidence index Ici is calculated for each of the colors red, orange, blue and purple in each image of the image sequence.

[0081] The instantaneous confidence is the segmentation luminous area Z of the current image of the image sequence. L First, the segmentation luminous region Z with a flashing state and a sufficient classification confidence index Icc is calculated based on the parameters of i. L i is taken into account.

[0082] Segmentation luminous region Z with classification confidence index Icc sufficient for consideration L To determine i, a state machine in the form of hysteresis is shown in FIG.

[0083] Light-emitting area Z L The state of i is initialized in the "off" state as follows: · If the classification confidence index Icc is greater than a predetermined threshold C3, a transition from state "off" to state "on" is made; If the classification confidence index Icc is less than a predetermined threshold C4, the transition from the "on" state to the "off" state is made. An example of the calculation of this instantaneous confidence index Ici for an image and a color is the following: L It can be the sum of the classification confidence index Icc of i. For example, for red, [Formula 3]

number

[0084] Then, to obtain the overall confidence index for each image in the image sequence and for each of the colors red, orange, blue, and purple, the equation: [Formula 4] ICG(t)=(1-α)*ICG(t-1)+α*Ici The instantaneous confidence is filtered out over a predetermined time period using Ici is the instantaneous confidence index of the color under consideration; α is a predetermined coefficient associated with the color and makes it possible to determine the weight of the instantaneous confidence index Ici in the calculation of the global confidence index ICG, for example the value of the coefficient α is between 0 and 1, preferably between 0.02 and 0.15.

[0085] The larger the value of the coefficient α, the greater the weight of the instantaneous reliability index Ici in the calculation of the overall reliability index ICG.

[0086] According to an exemplary embodiment, the coefficient α is determined by the segmentation light emitting area Z L It varies according to the parameter i. For example, the coefficient α is Multiple segmented luminous areas Z L is increased if i has a classification confidence index Icc greater than a predetermined threshold, If the computer 3 already indicates the presence of an emergency vehicle in the scene, the segmentation of the luminous area Z is reduced. L The method results can be stabilized and reinforced in the event of a momentary loss of detection in the i detection chain, which may be due to, for example, occlusion, excessive brightness or excessive measurement noise.

[0087] According to an exemplary embodiment, the coefficient α is determined by the segmentation luminous region Z L It varies as a function of the position of i. In particular, the coefficient α is The image is segmented into separate luminous regions Z located below the horizon H. L i is reduced (in other words, the integration speed is reduced), Multiple segmented luminous areas Z L If i is above the horizon H, it is increased (which corresponds to a high integration rate).

[0088] According to an exemplary embodiment, the coefficient α is determined by dividing the plurality of segmented luminous regions Z L i vary as a function of their relative positions. In particular, the multiple segmented luminous regions Z L If the i's are aligned on the same line L, the coefficient α is increased.

[0089] Varying the parameter α allows for adjusting the sensitivity of the method and therefore the segmentation luminous region Z L Depending on the parameter i, the event can be detected more or less quickly.

[0090] Also, for each segmentation luminous area Z of the current image of the image sequence L It is also possible to assign a weight to i that depends on other parameters, e.g. Segmentation luminous area Z L If i is small (i.e., has a size less than 20 pixels, for example) and / or is in the vicinity of a horizon H, the value of the weight is reduced; If the segmentation luminous area is not very bright (e.g., has a luminous intensity less than 1000 lx), the value of the weight is reduced; If the color of the light is not clear (for example, if it is saturated white light), the value of the weight is reduced; Segmentation luminous area Z L Other segmentation luminous regions Z that are similar (in terms of size, location, intensity, and color) to i L If it is in the neighborhood of i, the value of the weight is increased.

[0091] The weights are used to segment the light-emitting regions Z L If i have strongly correlated location, brightness, and color, the growth of the overall confidence index ICG can be accelerated. If the light size is small and the intensity is low, the false positive rate can be reduced by slowing down the growth of the overall confidence index ICG, but this leads to an acceptable detection delay for distant emergency vehicles.

[0092] Step 700 allows for a significant reduction in the false positive detection rate while still maintaining a sufficient detection rate for emergency vehicle flashing lights.

[0093] At the end of step 700, the computer 3 is able to indicate the presence of an emergency vehicle in the environment behind the vehicle 1, corresponding to the image sequence acquired by the camera 2, and in particular to determine the luminous area Z, for example using a hysteresis threshold known per se. L It can be declared that i is an emergency vehicle's flashing light.

[0094] A state machine for the hysteresis configuration is shown in FIG.

[0095] Light-emitting area Z L The state of i is initialized (Ei) in the "off" state as follows: If the overall confidence index ICG is greater than a predetermined threshold C1, a transition from the "off" state to the "on" state is made; If the overall confidence index ICG is less than a predetermined threshold C2, a transition from the state "on" to the state "off" is made.

[0096] In this case, the driver of the vehicle (if autonomous) or other type of vehicle 1 can take the necessary measures to facilitate and not hinder the movement of said emergency vehicle.

Claims

1. 1. A method for processing a video stream of images taken by at least one color camera (2) mounted on a motor vehicle (1), said images being used by a computer (3) mounted on said vehicle to detect priority vehicles (4) located in the environment of said vehicle (1), said at least one camera being aimed towards the rear of said vehicle, said method comprising: an acquisition step (100) for acquiring a sequence of images; For each image in the sequence of images: The colored light emitting area (Z Ｌ a segmentation step (200) in which a colorimetric segmentation based on thresholding is performed to enable detection of i) Each segmentation light emitting area (Z Ｌ i) and, according to the tracking, each luminous region segmented in the segmentation step (200) is divided into predicted luminous regions (Z Ｐ i), a tracking step (300) - Using a previously trained classifier, each segmented luminous region (Z Ｌ i) a classification step (400) performing a colorimetric classification; In order to be able to distinguish the flashing characteristics of the segmentation light emitting areas (Z L i), each segmentation light emitting area (Z Ｌ i) a frequency analysis step (500) for performing a frequency analysis; Segmentation light emitting area (Z Ｌ a calculation step (700) of calculating an overall confidence index (ICG) for each image of the image sequence in order to be able to declare i) to be a flashing light; A method comprising:

2. In the segmentation step (200), four categories are distinguished: ·red, ・Orange, blue, and ·violet, According to the light emitting region (Z Ｌ The method of claim 1 , wherein a predetermined segmentation threshold is used to segment i).

3. 3. The method according to claim 1 or 2, wherein after the segmentation step (200), the method further comprises a so-called post-segmentation filtering step (210) making it possible to filter out the results from the segmentation step (200), said post-segmentation filtering step (210) being performed according to predetermined criteria relating to position and / or size and / or color and / or intensity.

4. The post-segmentation filtering step (210) selects luminous regions (Z) located in image portions away from the horizon (H) and from the vanishing point (F) and having a size below a predetermined dimension threshold. Ｌ The method of claim 3, further comprising a dimension filtering substep (211) of filtering out i).

5. The post-segmentation filtering step (210) filters out luminous regions (Z) having a size exceeding a predetermined dimension threshold and a luminous intensity below a predetermined luminous intensity threshold. Ｌ The method of claim 3, further comprising the substep (212) of filtering out i).

6. The post-segmentation filtering step (210) filters out luminous regions (Z) located below a horizon (H) defined on the images of the image sequence. Ｌ 4. The method of claim 3, further comprising a position filtering substep (213) of filtering out i).

7. The post-segmentation filtering step (210) is performed by filtering the segmentation luminous regions (Z Ｌ The method of claim 3, further comprising the substep (215) of performing filtering based on directional colored thresholding on i).

8. The method further comprises: determining whether each segmentation luminous region (Z ) for which no association is found at the end of the tracking step (300) is associated with the segmentation luminous region (Z ); Ｌ The method according to any one of claims 1 to 7, further comprising a second segmentation step (310) for i).

9. The second segmentation step (310) comprises: a first substep (311) of expanding the segmentation threshold and repeating the segmentation step (200) and the tracking step (300) for each image of the image sequence with a newly expanded segmentation threshold corresponding to the color of the segmented luminous area; a second substep (312) of modifying the segmentation threshold to correspond to a white segmentation threshold if no associations have been found at the end of the first substep; The method of claim 8, which recites claim 2, comprising:

10. In the frequency analysis step (500), each segmentation luminous region (Z Ｌ i) flashing frequency is greater than a first frequency threshold (S Ｆ 1) and the first frequency threshold (S Ｆ a second frequency threshold (S Ｆ 2), where both thresholds are predetermined; The flashing frequency is greater than the first frequency threshold (S Ｆ 1) is smaller than the threshold value, and the segmentation light emitting area is not flashing or is only flashing weakly, and is therefore not considered to be a flashing light; The flashing frequency is greater than the second frequency threshold (S Ｆ 2) If the segmentation light emitting area is larger than the flashing light, the segmentation light emitting area is also not considered to be a flashing light. The segmentation luminous area (Z Ｌ i) is filtered out, 10. The method according to any one of claims 1 to 9.

11. The first frequency threshold (S Ｆ 1) is equal to 1 Hz, and the second frequency threshold (S Ｆ 11. The method of claim 10, wherein 2) is equal to 5 Hz.

12. The method further comprises: determining a displacement of each segmentation light emitting area (Z Ｌ The method according to any one of claims 1 to 11, comprising a directional analysis step (600) for performing a directional analysis of i).

13. The displacement direction obtained in the direction analysis step (600) is The segmentation light emitting area (Z Ｌ i) is immobile; The segmentation light emitting area (Z) is projected in a direction away from the vehicle (1) relative to the vehicle (1). Ｌ i) is moving, If it can be concluded that the segmentation luminous region (Z Ｌ i) is filtered out, 13. The method of claim 12.

14. A computer program which, when executed by a computer, an acquisition step (100) for acquiring a sequence of images; For each image in the sequence of images: The colored light emitting area (Z Ｌ a segmentation step (200) in which a colorimetric segmentation based on thresholding is performed to enable detection of i) Each segmentation light emitting area (Z Ｌ i) and, according to the tracking, each luminous region (Z Ｌ i) is the predicted light-emitting area of ​​the same color (Z Ｐ i), a tracking step (300) - Using a previously trained classifier, each segmented luminous region (Z Ｌ i) a classification step (400) performing a colorimetric classification; In order to be able to determine the flashing characteristics of the segmentation light emitting areas, each segmentation light emitting area (Z Ｌ i) a frequency analysis step (500) for performing a frequency analysis; Segmentation light emitting area (Z Ｌ a calculation step (700) of calculating an overall confidence index (ICG) for each image of the image sequence in order to be able to declare i) to be a flashing light; 10. A computer program comprising instructions for implementing a method comprising:

15. A vehicle (1) comprising at least one color camera (2) directed towards the rear of the vehicle and capable of acquiring a video stream of images of the environment behind the vehicle, and at least one computer (3), The computer (3) an acquisition step (100) for acquiring a sequence of images; For each image in the sequence of images: The colored light emitting area (Z Ｌ a segmentation step (200) in which a colorimetric segmentation based on thresholding is performed to enable detection of i) Each segmentation light emitting area (Z Ｌ i) and, according to the tracking, each luminous region segmented in the segmentation step (200) is divided into predicted luminous regions (Z Ｐ i), a tracking step (300) - Using a previously trained classifier, each segmented luminous region (Z Ｌ i) a classification step (400) performing a colorimetric classification; In order to be able to determine the flashing characteristics of the segmentation light emitting areas, each segmentation light emitting area (Z Ｌ i) a frequency analysis step (500) for performing a frequency analysis; Segmentation light emitting area (Z Ｌ a calculation step (700) of calculating an overall confidence index (ICG) for each image of the image sequence in order to be able to declare i) to be a flashing light; is configured to achieve Vehicle (1).

Images