In-vehicle camera image processing method

Abstract

The present invention belongs to the technical field of vehicle-mounted vision devices, and particularly relates to a method for processing images of vehicle-mounted cameras, which collects driving environment information and real-time image information, determines the driving scenario in which the vehicle is located according to a basic database, and adopts image configuration information matching the driving scenario. The present invention can timely change the image processing algorithm and camera configuration when the lighting conditions change, so that the vehicle-mounted camera can take into account the image quality in strong light and low light environments and obtain an image quality that meets the requirements of autonomous driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-mounted vision devices, and particularly relates to a method for processing vehicle-mounted camera images. Background Art

[0002] Vehicle-mounted cameras are indispensable devices for autonomous driving systems. The driving conditions of vehicles change rapidly, and the change in the brightness conditions where the vehicle is located will have a huge impact on the image quality of vehicle-mounted cameras. For example, when driving into an underground garage or entering and exiting a tunnel under the scorching sun, where the environmental brightness changes suddenly, the images collected by existing vehicle-mounted cameras cannot accurately distinguish obstacles. The driving strategy judgment of vehicle autonomous driving depends on the images collected by vehicle-mounted cameras. If the image quality of vehicle-mounted cameras cannot be guaranteed in the case of changing environmental brightness, it will bring serious safety hazards to autonomous driving. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for processing vehicle-mounted camera images that can ensure image quality in the case of changing brightness.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] A method for processing vehicle-mounted camera images,

[0006] Simulate driving in a laboratory standard environment, an indoor environment, an outdoor environment, and an extreme environment respectively, and measure the image configuration information that meets the driving safety requirements under various road conditions under different illuminance and color temperature conditions to form a basic database;

[0007] After the vehicle starts, collect driving environment information and real-time image information, and judge the current driving scenario of the vehicle according to the basic database and adopt the corresponding image configuration information;

[0008] When the vehicle is driving, collect driving environment information and real-time image information, and judge whether the driving scenario of the vehicle has changed according to the basic database. If the driving scenario changes, update the image configuration information that matches the driving scenario.

[0009] Compared with the prior art, the present invention has the following technical effects: It can timely change the image processing algorithm and camera configuration when the lighting conditions change, so that the vehicle-mounted camera can take into account the image quality in strong light and weak light environments and obtain the image quality that meets the requirements of autonomous driving. Brief Description of the Drawings

[0010] The following briefly describes the content expressed by each drawing in this specification and the marks in the drawings:

[0011] Figure 1 is the schematic diagram of the present invention;

[0012] Figure 2 This is a schematic diagram of the present invention. Specific Embodiments

[0013] The following further elaborates on the specific embodiments of the present invention in conjunction with the accompanying drawings through the description of embodiments.

[0014] A method for processing the on-vehicle camera image

[0015] During driving simulations in a laboratory standard environment, an indoor environment, an outdoor environment, and an extreme environment respectively, under different illuminance and color temperature conditions, the image configuration 50 information that meets the driving safety requirements under various road conditions is measured and obtained to form a basic database 10.

[0016] After the vehicle starts, the driving environment information 30 and the real-time image information 40 are collected, and based on the basic database 10, the current driving scenario 20 of the vehicle is determined and the corresponding image configuration 50 information is adopted.

[0017] When the vehicle is driving, the driving environment information 30 and the real-time image information 40 are collected, and based on the basic database 10, it is determined whether the driving scenario 20 of the vehicle has changed. If the driving scenario 20 changes, the image configuration 50 information matching the driving scenario 20 is updated.

[0018] When constructing the basic database 10, it is necessary to simulate as many different driving scenarios 20 as possible to obtain the image configuration 50 information with image quality meeting the driving safety requirements under the corresponding driving scenarios 20. The image configuration 50 information includes the adjustment thresholds, tolerance intervals, and combined variable parameter adjustment ranges of basic configurations such as brightness, grayscale, color temperature, white balance, contrast, exposure, color, sharpness, noise reduction, gain, and various corrections and compensations.

[0019] Under different driving scenarios 20, the external environment of the vehicle will affect the quality of the real-time image. When the vehicle is in different driving states, the requirements for the real-time image quality are different. Therefore, in this embodiment, it is necessary to collect the driving environment information 30 and the real-time image information 40 for determining the driving scenario 20. Among them, the driving environment information 30 includes external environment information 31 and vehicle driving information 32;

[0020] The external environment information 31 includes illuminance, temperature, rain and fog, time, weather broadcast,

[0021] The vehicle driving information 32 includes vehicle speed, headlight on / off state, vehicle location, road information, recognizable road signs.

[0022] Division of driving scenario 20. According to one or more pieces of information among time, illuminance, weather broadcast, headlight on state, and real-time video information 40, the driving scenario 20 is divided into a natural high-illuminance scenario 21, a natural low-illuminance scenario 22, a light high-illuminance scenario 23, a light low-illuminance scenario 24, and an illuminance change scenario 25 according to the light source type, illuminance level, illuminance change rate, and the road where the vehicle is located; the illuminance change scenario 25 includes a regular illuminance change scenario 25a and an irregular illuminance change scenario 25b.

[0023] When the vehicle is driving, the driving environment information 30 and the real-time video information 40 are monitored in real time. When the light source changes, the illuminance change value exceeds the threshold, or the illuminance change rate exceeds the threshold, it is determined that the driving scenario 20 has changed. The driving environment information 30 and the real-time video information 40 are compared with the pre-stored information in the basic database 10 to determine the current driving scenario 20, and the screen configuration 50 matching the driving scenario 20 is updated.

[0024] The method for obtaining the real-time video information 40 is as follows:

[0025] A. Divide the collected image into N columns × M rows of regions;

[0026] B. Obtain the image quality parameters of the images in each divided region;

[0027] C1. Compare the image quality parameters of the image in a certain divided region of the current frame with the image quality parameters of the same divided region of the next frame to obtain the image quality parameters 41 of each region;

[0028] C2. Compare the image quality parameters of the current frame with the image quality parameters of the next frame to obtain the frame image quality parameters 42.

[0029] When obtaining the real-time video information 40,

[0030] In step A, several regions located at the center of the image are defined as the main viewing areas. In this embodiment, the three columns of regions in the middle of the image are the initial main viewing areas;

[0031] In step C1, if the change value of the image quality parameters of a certain region exceeds the threshold, that region is defined as the main viewing area;

[0032] When the vehicle is driving, the real-time video information 40 is collected at equal time intervals, and the collection time interval of the main viewing area is less than that of the non-main viewing area.

[0033] The frame image quality parameters 42 include the average brightness and its change value, average color temperature and its change value, average gray scale and its change value, and average contrast and its change value of the frame image,

[0034] The regional picture quality parameter 41 includes the brightness of the picture within the region and its change value, the color temperature and its change value, the gray scale and its change value, and the contrast and its change value.

[0035] The method for judging the change of the driving environment based on the picture analysis data is as follows:

[0036] D1. If the measured frame picture quality parameter 42 and the regional picture quality parameters 41 of each region are all within the threshold range of the corresponding scene in the basic database 10, it is determined that there is no change in the current driving scene 20, and there is no need to change the picture configuration 50.

[0037] D2. If the measured frame picture quality parameter 42 and the regional picture quality parameters 41 of each region all exceed the threshold range of the corresponding scene in the basic database 10, go to step E.

[0038] E. Redefine several regions located in the center of the picture as the main viewing areas, calculate the weighted regional picture quality parameters 41 of the main viewing areas and compare them with the regional picture quality parameters 41 outside the main viewing areas.

[0039] If the difference between the regional picture quality parameter 41 of the main viewing area and that outside the main viewing area is within the threshold range of the corresponding scene in the basic database 10, it is determined that there is no change in the current driving scene 20, and adjust the picture configuration 50 of the area where the change value in the main viewing area exceeds the threshold range.

[0040] If the difference between the regional picture quality parameter 41 of the main viewing area and that outside the main viewing area exceeds the threshold range of the corresponding scene in the basic database 10, combine the driving environment information 30 and the real-time picture information 40 to judge what kind of driving scene 20 the current vehicle is in according to the basic database 10, and change the picture configuration 50.

[0041] D3. If the measured frame picture quality parameter 42 is within the threshold range of the corresponding scene in the basic database 10, and the regional picture quality parameter 41 of a certain region exceeds the threshold range of the corresponding scene in the basic database 10, go to step F.

[0042] F. Judge in the regions where the regional picture quality parameter 41 exceeds the threshold range of the corresponding scene in the basic database 10:

[0043] Whether the change values of brightness and color temperature show periodic changes?

[0044] Within the time period T, whether the change value of color temperature is less than the change value of brightness?

[0045] On what kind of road is the vehicle driving?

[0046] Is the driving time of the vehicle during the day?

[0047] If both the brightness and color temperature change rates show periodic changes, and the color temperature change rate is less than the brightness change rate, and the vehicle is driving during the day on an outdoor road, it is determined that the vehicle is in the illumination regular change scenario 25a; among the trees

[0048] If both the brightness and color temperature change rates show periodic changes, and the color temperature change rate is greater than or equal to the brightness change rate, and the vehicle is located in a tunnel or an indoor road, it is determined that the vehicle is in the illumination regular change scenario 25a; inside the tunnel

[0049] If the brightness and color temperature change rates change irregularly, and the color temperature change rate is less than the brightness change rate, and the vehicle is driving during the day on an outdoor road, it is determined that the vehicle is in the illumination irregular change scenario 25b; under the shadow of a building

[0050] If the brightness and color temperature change rates change irregularly, and the color temperature change rate is greater than or equal to the brightness change rate, and the vehicle is driving at night on an outdoor road, it is determined that the vehicle is in the illumination irregular change scenario 25b; on an urban street

[0051] Then, the screen configuration 50 of the basic database 10 is changed according to the driving scenario 20.

[0052] When the vehicle is in the illumination regular change scenario 25a, for example, under the shadow of regularly arranged roadside trees,

[0053] If the color temperature change rate is less than the brightness change rate, the average brightness of the screen is extracted from the frame image quality parameter 42, and this is used as the basic variable of the dynamic brightness. The dynamic strength and weakness distribution area of the brightness, the ratio of the brightness values, and the rate of change of the brightness value with the vehicle speed are calculated. The exposure is compensated for the low brightness area, and the contrast of the screen is reduced; if the color temperature change rate is greater than or equal to the brightness change rate, the average color temperature of the screen is extracted from the frame image quality parameter 42, and this is used as the basic variable of the dynamic color temperature. The dynamic distribution area of the color temperature, the ratio of the color temperature values, and the rate of change of the color temperature value with the vehicle speed are calculated, and the color is corrected and the screen is balanced;

[0054] When the vehicle is in the illumination irregular change scenario 25b, for example, under the shadow of irregular urban buildings, the threshold range of the image quality parameters in this area is increased, and this area is processed for strengthening white balance, suppressing noise, correcting color, balancing exposure, and suppressing bright spots.

[0055] In the in-vehicle camera image adjustment device used in this embodiment, the camera 100 is connected to the environmental perception component 200 for collecting driving environment information 30. The camera 100 includes an image sensor 110 and an image processor 120. The image processor 120 receives the image uploaded by the image sensor 110, analyzes it to obtain real-time image information 40, and the image processor 120 receives the driving environment information 30 uploaded by the environmental perception component 200. By combining the driving environment information 30 and the real-time image information 40, it determines the driving scenario in which the vehicle is located, selects the corresponding image configuration 50 to process the original image collected by the image sensor 110, and then outputs the image data stream.

[0056] Inside the image processor 120, there are a frame analysis module 121 for analyzing the image uploaded by the image sensor 110, a scenario judgment module 122 for receiving the driving environment information 30 and the real-time image information 40 and judging the driving scenario 20 in which the vehicle is located, a storage module 123 for storing the image configurations 50 in each driving scenario 20, and a frame processing module 124 for adjusting the image configuration 50.

[0057] The environmental perception component 200 includes a temperature sensor 210 for judging the external physical environment of the vehicle, an illuminance sensor 220, and a rain and fog sensor 230; it also includes

[0058] a vehicle speed sensor 240 for obtaining the driving speed of the vehicle,

[0059] a headlight switch recognition unit 250 for obtaining the on / off state of the vehicle headlights,

[0060] a road condition information unit 260 for obtaining the location of the vehicle, the type of road, and identifying road signs,

[0061] a clock unit 270 for obtaining the time;

[0062] a weather broadcast unit 280 for obtaining weather broadcast information.

Claims

1. A method for processing the images of a vehicle-mounted camera, characterized in that: Simulate driving in a laboratory standard environment, an indoor environment, an outdoor environment, and an extreme environment respectively, and measure the image configuration (50) information that meets the driving safety requirements under various road conditions under different illuminance and color temperature conditions to form a basic database (10); After the vehicle starts, collect driving environment information (30) and real-time image information (40), and determine the current driving scenario (20) of the vehicle according to the basic database (10) and adopt the corresponding image configuration (50) information; When the vehicle is driving, collect driving environment information (30) and real-time image information (40), and determine whether the driving scenario (20) where the vehicle is located changes according to the basic database (10). If the driving scenario (20) changes, update the image configuration (50) information that matches the driving scenario (20); The method for obtaining the real-time image information (40) is as follows: A. Divide the collected image into N columns × M rows of regions, and define several regions located in the center of the image as the first main viewing area; When the vehicle is driving, collect real-time image information (40) at equal time intervals, and the collection time interval of the first main viewing area is less than that of the non-first main viewing area; B. Obtain the image quality parameters of the images in each divided region; C1. Compare the image quality parameters of the image in a certain divided region of the current frame with the image quality parameters of the same divided region of the next frame to obtain the image quality parameters (41) of each region. The regional image quality parameters (41) include the brightness and its change value, color temperature and its change value, gray level and its change value, and contrast and its change value of the image in the region; If the change value of the image quality parameters of a certain region exceeds the threshold, define this region as the second main viewing area; C2. Compare the image quality parameters of the current frame with the image quality parameters of the next frame to obtain the frame image quality parameters (42). The frame image quality parameters (42) include the average brightness and its change value, average color temperature and its change value, average gray level and its change value, and average contrast and its change value of the frame image; The method for judging the change of the driving scenario based on the image analysis data is as follows: D1. If the measured frame image quality parameters (42) and the regional image quality parameters (41) of each region are all within the threshold range of the corresponding scenario in the basic database (10), it is determined that the current driving scenario (20) has not changed and there is no need to change the image configuration (50); D2. If the measured frame image quality parameters (42) and the regional image quality parameters (41) of each region all exceed the threshold range of the corresponding scenario in the basic database (10), go to step E; E. Redefine several regions located in the center of the image as the first main viewing area, and compare the weighted calculated regional image quality parameters (41) of the first main viewing area with the regional image quality parameters (41) of the non-first main viewing area; If the difference between the regional image quality parameters (41) of the first main viewing area and the non-first main viewing area is within the threshold range of the corresponding scenario in the basic database (10), it is determined that the current driving scenario (20) has not changed, and adjust the image configuration (50) of the regions in the first main viewing area where the change value exceeds the threshold range; If the difference between the regional picture quality parameters (41) of the main viewing area one and the main non-main viewing area one exceeds the threshold range of the corresponding scenario in the basic database (10), combine the driving environment information (30) and the real-time picture information (40) to determine what driving scenario (20) the current vehicle is in according to the basic database (10), and change the picture configuration (50).

2. The vehicle-mounted camera image processing method according to claim 1, wherein: The driving environment information (30) includes external environment information (31) and vehicle driving information (32); The external environment information (31) includes illuminance, temperature, rain and fog, time, weather broadcast, The vehicle driving information (32) includes vehicle speed, headlight on state, vehicle location, road information, recognizable road signs.

3. The method for processing the on-vehicle camera image according to claim 1, wherein: The driving scenario (20) is divided into a natural light high illuminance scenario (21), a natural light low illuminance scenario (22), a light high illuminance scenario (23), a light low illuminance scenario (24), and an illuminance change scenario (25) according to the light source type, illuminance level, illuminance change rate, and the road where the vehicle is located; the illuminance change scenario (25) includes an illuminance regular change scenario (25a) and an illuminance irregular change scenario (25b).

4. The vehicle-mounted camera image processing method according to claim 3, wherein: When the vehicle is driving, continuously monitor the driving environment information (30) and the real-time picture information (40). When the light source changes, the illuminance change value exceeds the threshold, or the illuminance change rate exceeds the threshold, it is determined that the driving scenario (20) has changed. Compare the driving environment information (30), the real-time picture information (40) with the pre-stored information in the basic database (10) to determine the current driving scenario (20), and update the picture configuration (50) matching the driving scenario (20).

5. The method for processing the on-vehicle camera image according to claim 1, wherein: The method for judging the change of the driving environment based on the picture analysis data further includes,[[]] D3. If the measured frame picture quality parameter (42) is within the threshold range of the corresponding scenario in the basic database (10), and the regional picture quality parameter (41) of a certain area exceeds the threshold range of the corresponding scenario in the basic database (10), go to step F; F. Judge in the area where the regional picture quality parameter (41) exceeds the threshold range of the corresponding scenario in the basic database (10), Whether the change values of brightness and color temperature show periodic changes; Whether the color temperature change value is less than the brightness change value within the time period T; On what kind of road the vehicle is driving; Whether the driving time of the vehicle is during the day; If both the brightness and color temperature change rates show periodic changes, the color temperature change rate is less than the brightness change rate, and the vehicle is driving during the day and is on an outdoor road, it is determined that the vehicle is in the illuminance regular change scenario (25a); If both the brightness and color temperature change rates show periodic changes, the color temperature change rate is greater than or equal to the brightness change rate, and the vehicle is in a tunnel or an indoor road, it is determined that the vehicle is in the illuminance regular change scenario (25a); If the brightness and color temperature change rates change irregularly, the color temperature change rate is less than the brightness change rate, and the vehicle is driving during the day and is on an outdoor road, it is determined that the vehicle is in the illuminance irregular change scenario (25b); If the brightness and color temperature change rates change irregularly, the color temperature change rate is greater than or equal to the brightness change rate, and the vehicle is driving at night and is on an outdoor road, it is determined that the vehicle is in the illuminance irregular change scenario (25b); Change the screen configuration (50) of the driving scenario (20) according to the basic database (10).

6. The vehicle-mounted camera image processing method according to claim 5, wherein: When the vehicle is in the scene (25a) with regularly changing illuminance, if the color temperature change rate is less than the luminance change rate, extract the average luminance of the screen from the frame image quality parameters (42), use this as the basic variable of the dynamic luminance, calculate the distribution area of the dynamic strength and weakness of the luminance, the ratio of the luminance values, and the rate of change of the luminance value with the vehicle speed, compensate for the exposure of the low-luminance area, and reduce the contrast of the screen; if the color temperature change rate is greater than or equal to the luminance change rate, extract the average color temperature of the screen from the frame image quality parameters (42), use this as the basic variable of the dynamic color temperature, calculate the distribution area of the dynamic color temperature, the ratio of the color temperature values, and the rate of change of the color temperature value with the vehicle speed, correct the color, and balance the screen; When the vehicle is in the scene (25b) with irregularly changing illuminance, increase the threshold range of the image quality parameters in this area, and perform processing on this area such as strengthening white balance, suppressing noise, correcting color, balancing exposure, and suppressing bright spots.

Images