Perception method for vehicle-mounted camera

Abstract

The invention discloses a perception method for a vehicle-mounted camera, and the method comprises the steps: obtaining an image collected by a vehicle-mounted front-view camera in real time, and judging that the vehicle-mounted front-view camera is in a state of being directly irradiated by strong light according to whether the range of a single pixel value in the image exceeds a set threshold value or not; the method comprises the following steps: recording a current frame image and a previous frame image of a vehicle-mounted foresight camera which are directly irradiated by strong light, and calculating a line pixel change difference between the current frame image and the previous frame image; and correcting the image irradiated by the strong light by using the calculated pixel change difference value so as to carry out brightness reduction processing on the current frame image. According to the invention, the detection accuracy and intelligence of the vehicle-mounted camera can be improved, and the use safety of the whole vehicle is improved.

Description

Technical Field

[0001] This invention relates to the technical field of camera sensing and control, and more specifically, to a sensing method for an in-vehicle camera. Background Technology

[0002] As vehicles become increasingly intelligent, they need to be able to "see" their surroundings like humans, making cameras a crucial sensor. Cameras collect optical information about the vehicle's surroundings and convert it into digital data, providing autonomous driving systems with rich image information about roads, vehicles, pedestrians, and traffic signs. Furthermore, in conjunction with other sensors such as LiDAR and millimeter-wave radar, cameras can achieve more comprehensive perception capabilities, helping intelligent driving vehicles navigate safely and efficiently in complex traffic environments. However, when a vehicle is driving towards direct sunlight or when oncoming traffic has strong headlights, camera perception faces multiple challenges: Drastic changes in light can cause overexposure or underexposure of the camera image, resulting in blurry images, loss of detail, and difficulty in accurately identifying obstacles, lane lines, or traffic signals. Moreover, due to motion blur and low light, high-speed driving can easily lead to motion blur due to shutter speed limitations, affecting target detection accuracy. In summary, when a vehicle's camera encounters strong light, it can become overexposed, resulting in an overly bright image, leading to inaccurate perception of the surrounding environment and affecting image detection results. Therefore, improving the intelligence and accuracy of camera perception and avoiding the influence of strong light is of great significance. Summary of the Invention

[0003] This invention provides a sensing method for vehicle-mounted cameras, which solves the problems of inaccurate image detection results, overexposure, and overbrightness caused by strong light exposure in existing vehicle-mounted cameras. It can improve the accuracy and intelligence of vehicle-mounted camera detection and increase the safety of the entire vehicle.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A sensing method for an in-vehicle camera, comprising:

[0006] The system acquires images captured by the vehicle's front-view camera in real time, and determines whether the range of a single pixel value in the image exceeds a set threshold. If so, it determines that the vehicle's front-view camera is under direct strong light.

[0007] Record the current frame image and the previous frame image of the vehicle-mounted front-view camera when it is directly exposed to strong light, and calculate the difference in pixel change between the current frame image and the previous frame image.

[0008] The calculated pixel change difference is used to correct the image after it has been illuminated by strong light, so as to reduce the brightness of the current frame image.

[0009] Preferably, the step of determining whether the range of a single pixel value in the image exceeds a set threshold, and if so, determining that the vehicle-mounted forward-facing camera is under direct strong light, includes:

[0010] When a single pixel value in an image exceeds 200, the corresponding pixel is judged to be in a glare state. When the pixels corresponding to the glare state account for more than 30% of all pixels, the entire camera is considered to be in a state of being directly exposed to strong light.

[0011] Preferably, the step of calculating the difference in row pixel changes between the current frame image and the previous frame image includes:

[0012] The number of pixels occupied by any object in the current frame image is N1, and the sum of the values ​​of the N1 pixels is M1;

[0013] The number of pixels occupied by any object in the previous frame image is N2, and the sum of the values ​​of the N2 pixels is M2;

[0014] The pixel difference V3 is calculated using the formula V3=M2 / N2-M1 / N1, and is used as the corresponding pixel change difference.

[0015] Preferred options also include:

[0016] The pixel change difference is calculated using multiple frames preceding the current frame image to correct the pixel change difference corresponding to the current frame image.

[0017] Preferred options also include:

[0018] When the vehicle's front-view camera is directly exposed to strong light, other vehicle cameras and the front-view camera simultaneously capture images of the same obstacle. In a certain frame, the pixel values ​​of the two different cameras capturing the same obstacle are compared, and the pixel change difference between the two cameras is calculated. Then, the calculated pixel change difference is used to correct the image captured after being exposed to strong light.

[0019] Preferably, the step of simultaneously capturing images of the same obstacle using other vehicle-mounted cameras and a forward-facing vehicle-mounted camera includes:

[0020] Use the vehicle's rearview camera and frontview camera to simultaneously capture images of the same sky, trees, railings, or road surface.

[0021] Preferably, the calculation of the pixel change difference between the two cameras includes:

[0022] The sum of pixel values ​​V4 of N3 sky pixels in the image captured by the vehicle's front-view camera and the sum of pixel values ​​V5 of N4 sky pixels in the image detected by the rear-view camera are obtained. The average difference V6 is calculated according to the formula V6=V4 / N3-V5 / N4, and the average difference V6 is used as the corresponding pixel change difference for image correction.

[0023] Preferred options also include:

[0024] When the vehicle's forward-facing camera is directly exposed to strong light, the pixel change difference is calculated using different frame images of the same obstacle taken by different cameras. The calculated pixel change difference is then used to correct the image taken after being exposed to strong light.

[0025] Preferably, the step of calculating the pixel change difference using different frame images of the same obstacle captured by different cameras includes:

[0026] Images of the same obstacle are captured using both the vehicle's front-view camera and rear-view camera.

[0027] Get the sum of pixel values ​​V7 of N5 pixels in the image of the obstacle captured by the vehicle's front-view camera, and the sum of pixel values ​​V8 of N6 pixels in the image of the obstacle captured by the vehicle's rear-view camera in any subsequent frame.

[0028] The average difference V9 is calculated using the formula V9=V7 / N5-V8 / N6, and the average difference V6 is used as the corresponding pixel change difference for image correction.

[0029] Preferred options also include:

[0030] When an obstacle is detected in front of the vehicle, the detection information is recorded and stored, and then searched in images within a set frame range of other cameras to determine if there is a similar obstacle.

[0031] If so, it is considered the same obstacle, and the pixel change difference is calculated using different frame images of the same obstacle taken by different cameras. The calculated pixel change difference is then used to correct the image taken after being illuminated by strong light.

[0032] This invention provides a sensing method for vehicle-mounted cameras. It sets conditions and thresholds for strong light illumination on the camera, acquires images from the vehicle-mounted camera, and determines whether the current frame meets the threshold condition. If so, it calculates the pixel change difference between the current frame and the previous frame and uses this calculated difference to correct the image after strong light illumination. This solves the problems of inaccurate image detection results, overexposure, and overbrightness caused by strong light illumination in existing vehicle-mounted cameras, improving the accuracy and intelligence of vehicle-mounted camera detection and increasing the safety of the entire vehicle. Attached Figure Description

[0033] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below.

[0034] Figure 1 This is a schematic diagram of a sensing method for an in-vehicle camera provided by the present invention.

[0035] Figure 2 This is a schematic diagram of a process for improving the accuracy of a vehicle-mounted camera when it is exposed to strong light, provided by an embodiment of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and implementation methods.

[0037] To address the problems of inaccurate image detection results, overexposure, and excessive brightness caused by strong light exposure in existing vehicle-mounted cameras, this invention provides a sensing method for vehicle-mounted cameras. This method solves the problems of inaccurate image detection results, overexposure, and excessive brightness caused by strong light exposure in existing vehicle-mounted cameras, improves the accuracy and intelligence of vehicle-mounted camera detection, and increases the safety of vehicle use.

[0038] like Figure 1 and Figure 2 As shown, a sensing method for an in-vehicle camera includes:

[0039] S1: Real-time acquisition of images captured by the vehicle-mounted front-view camera, and determination of whether the range of a single pixel value in the image exceeds a set threshold. If so, the vehicle-mounted front-view camera is under direct strong light.

[0040] S2: Record the current frame image and the previous frame image of the vehicle-mounted front-view camera when it is directly exposed to strong light, and calculate the difference in pixel change between the current frame image and the previous frame image.

[0041] S3: The calculated pixel change difference is used to correct the image after being illuminated by strong light, so as to reduce the brightness of the current frame image.

[0042] Specifically, such as Figure 2 As shown, the method sets conditions and thresholds for strong light exposure of the camera, acquires images from the vehicle-mounted camera, and determines whether the current frame meets the threshold condition. If so, it is determined that the camera is under strong light exposure. The method then acquires image information from the current frame and the previous frame, calculates the pixel change difference between the two images, and uses this calculated difference to correct the image after strong light exposure. This method improves the accuracy and intelligence of vehicle-mounted camera detection, increasing the safety of the entire vehicle.

[0043] Furthermore, the step of determining whether the range of a single pixel value in the image exceeds a set threshold, and if so, determining that the vehicle-mounted forward-facing camera is under direct strong light, includes:

[0044] When a single pixel value in an image exceeds 200, the corresponding pixel is judged to be in a glare state. When the pixels corresponding to the glare state account for more than 30% of all pixels, the entire camera is considered to be in a state of being directly exposed to strong light.

[0045] Specifically, a vehicle can obtain a frame of its surrounding environment image through a camera. This image is composed of countless pixels; for example, a 1920*1080 resolution image has 1920*1080 pixels. Each pixel value ranges from 0 to 255. Higher values ​​are closer to white, and are considered bright. Lower values ​​are closer to black, and are considered dark. When a camera is directly exposed to strong light, its pixel value will approach 255, meaning that single pixel is bright. A threshold can be set; for example, if a single pixel value exceeds 200, it is considered to be in a glare state, meaning that pixel is very bright. When glare-prone pixels account for more than 30% of all pixels, we consider the entire camera to be under direct strong light.

[0046] Furthermore, the calculation of the difference in row pixel changes between the current frame image and the previous frame image includes:

[0047] The number of pixels occupied by any object in the current frame image is N1, and the sum of the values ​​of the N1 pixels is M1;

[0048] The number of pixels occupied by any object in the previous frame image is N2, and the sum of the values ​​of the N2 pixels is M2;

[0049] The pixel difference V3 is calculated using the formula V3=M2 / N2-M1 / N1, and is used as the corresponding pixel change difference.

[0050] In one embodiment, when a camera is in a scene illuminated by strong light, the image at this moment is recorded as Img2, and the previous frame of image data from this camera is recorded as Img1. Then, it is necessary to compare the detected objects in Img1 and Img2. For example, if a car is detected in Img1 and recorded as C1, the same car will be detected in Img2 and recorded as C2. Since the camera is illuminated by strong light in Img2, the image portion containing C2 will have brighter pixels than the image portion containing C1. Therefore, the specific changes in pixel values ​​after the image is illuminated by strong light can be determined by calculating the brightness difference between C1 and C2.

[0051] For example: Car C1 in Img1 occupies N1 pixels, and the same car C2 in Img2 occupies N2 pixels.

[0052] Let M1 be the sum of the pixel values ​​of N1 pixels. Then, let V1 be the average value of a single pixel. We know that V1 = M1 / N1.

[0053] Similarly, the same process can be performed on the same car C2 in Img2. The sum of the pixel values ​​of N2 pixels is M2, and the average single cluster value is denoted as V2. At this time, V2 = M2 / N2.

[0054] At this point, let V3 be the pixel difference between the same target cars C1 and C2 detected in images Img1 and Img2, where V3 = V2 - V1.

[0055] The pixel value change of the target vehicle can be used to represent the pixel change of the entire image. For example, in Img2, V3 can be subtracted from the pixel value of each 1920*1080 pixel in the image to reduce the brightness of the Img2 image, thereby solving the problem of glare caused by the camera being directly exposed to strong light, which leads to inaccurate detection.

[0056] The method further includes: calculating the pixel change difference using multiple frames preceding the current frame image to correct the pixel change difference corresponding to the current frame image.

[0057] In practical applications, extracting the previous frame from the same camera can lead to inaccurate calculation of brightness difference, as there is only one frame's difference. However, it is possible to calculate the difference across multiple frames and iteratively refine the V3 value.

[0058] The method also includes: when the vehicle-mounted front-view camera is directly exposed to strong light, using other vehicle-mounted cameras and the vehicle-mounted front-view camera to simultaneously capture images of the same obstacle, and comparing the pixel values ​​of the two different cameras capturing the same obstacle in a certain frame, calculating the pixel change difference between the two cameras, and then using the calculated pixel change difference to correct the image captured after being exposed to strong light.

[0059] Furthermore, the method of simultaneously capturing images of the same obstacle using other vehicle-mounted cameras and the vehicle-mounted front-view camera includes: simultaneously capturing images of the same sky, trees, railings, or road surface using the vehicle-mounted rear-view camera and the vehicle-mounted front-view camera.

[0060] Furthermore, the calculation of the pixel change difference between the two cameras includes: obtaining the sum of pixel values ​​V4 of N3 sky pixels in the image captured by the vehicle's front-view camera and the sum of pixel values ​​V5 of N4 sky pixels in the image detected by the rear-view camera; calculating the average difference V6 according to the formula V6=V4 / N3-V5 / N4; and using the average difference V6 as the corresponding pixel change difference for image correction.

[0061] In one embodiment, suppose a vehicle is driving towards the sun on an empty road. The front-view camera is illuminated by strong sunlight, and its image is denoted as Img3. Image Img3 exhibits glare, resulting in excessively high pixel values. The rear-view camera image in the same frame is denoted as Img4, which is in a normal state with normal pixel values. The difference in pixel values ​​can be calculated by comparing the brightness of the same objects in images Img3 and Img4. This difference can then be used to reduce the brightness of Img3. For example, if the front-view camera displays the sky above the image, and the rear-view camera also detects the same sky, the sum of the pixel values ​​of the N3 sky pixels detected by the front-view camera in a certain frame is V4, while the sum of the pixel values ​​of the N4 sky pixels detected by the rear-view camera is V5. The average difference is denoted as V6, and V6 = V4 / N3 - V5 / N4. The average difference V6 at this point can be applied to the entire image, that is, the pixel value of all pixels in Img3 minus V6, to correct the image from the front-view camera.

[0062] Similarly, obstacles that can be detected by front and rear cameras in the same frame, such as trees, roads, and railings, can be used to assist in the judgment.

[0063] The method also includes: when the vehicle-mounted forward-view camera is directly exposed to strong light, calculating the pixel change difference using different frame images of the same obstacle taken by different cameras, and using the calculated pixel change difference to correct the image taken after being exposed to strong light.

[0064] Furthermore, the calculation of pixel change differences using different frame images of the same obstacle captured by different cameras includes:

[0065] Images of the same obstacle are captured using both the vehicle's front-view camera and rear-view camera.

[0066] Get the sum of pixel values ​​V7 of N5 pixels in the image of the obstacle captured by the vehicle's front-view camera, and the sum of pixel values ​​V8 of N6 pixels in the image of the obstacle captured by the vehicle's rear-view camera in a later frame.

[0067] The average difference V9 is calculated using the formula V9=V7 / N5-V8 / N6, and the average difference V6 is used as the corresponding pixel change difference for image correction.

[0068] In one embodiment, when driving on an open road facing the sun, the forward-facing camera is illuminated by strong sunlight, denoted as glare state Img5. Assuming there is another target vehicle in an adjacent lane, the forward-facing camera detects its information and records the sum of the pixel values ​​V7 of the target vehicle's N5 pixels f. When overtaking, the rear-facing camera also detects the same vehicle in a certain frame, and records the sum of the pixel values ​​V8 of the target vehicle's N6 pixels in the image. The average pixel difference is recorded as V9, where V9 = V7 / N5 - V8 - N6. The glare image in Img5 can be adjusted using the difference V9; that is, subtracting V9 from the pixel value of each pixel in Img5 solves the glare problem.

[0069] The method further includes: when an obstacle is detected in front of the vehicle, the detection information is recorded and stored, and the images within a set frame range of other cameras are searched to determine whether there is a similar obstacle; if so, it is regarded as the same obstacle, and the pixel change difference is calculated using different frame images of the same obstacle taken by different cameras, and the calculated pixel change difference is used to correct the image taken after being illuminated by strong light.

[0070] As can be seen, this invention provides a perception method for vehicle-mounted cameras. It sets conditions and thresholds for strong light irradiation of the camera, acquires images from the vehicle-mounted camera, determines whether the current frame meets the threshold condition, and if so, calculates the pixel change difference between the current frame and the previous frame, using the calculated pixel change difference to correct the image after strong light irradiation. This solves the problems of inaccurate image detection results, overexposure, and overbrightness caused by strong light irradiation in existing vehicle-mounted cameras, improving the accuracy and intelligence of vehicle-mounted camera detection and increasing the safety of the entire vehicle.

[0071] The structure, features, and effects of the present invention have been described in detail above with reference to the embodiments shown in the figures. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, shall be within the protection scope of the present invention as long as they do not exceed the spirit covered by the specification and figures.

Claims

1. A sensing method for an in-vehicle camera, characterized in that, include: The system acquires images captured by the vehicle's front-view camera in real time, and determines whether the range of a single pixel value in the image exceeds a set threshold. If so, it determines that the vehicle's front-view camera is under direct strong light. Record the current frame image and the previous frame image of the vehicle-mounted front-view camera when it is directly exposed to strong light, and calculate the difference in pixel change between the current frame image and the previous frame image. The calculated pixel change difference is used to correct the image after it has been illuminated by strong light, so as to reduce the brightness of the current frame image.

2. The sensing method for an in-vehicle camera according to claim 1, characterized in that, The step of determining whether the range of a single pixel value in the image exceeds a set threshold, and if so, determining that the vehicle-mounted forward-facing camera is under direct strong light, includes: When a single pixel value in an image exceeds 200, the corresponding pixel is judged to be in a glare state. When the pixels corresponding to the glare state account for more than 30% of all pixels, the entire camera is considered to be in a state of being directly exposed to strong light.

3. The sensing method for an in-vehicle camera according to claim 2, characterized in that, The calculation of the difference in row pixel changes between the current frame image and the previous frame image includes: The number of pixels occupied by any object in the current frame image is N1, and the sum of the values ​​of the N1 pixels is M1; The number of pixels occupied by any object in the previous frame image is N2, and the sum of the values ​​of the N2 pixels is M2; The pixel difference V3 is calculated using the formula V3=M2 / N2-M1 / N1, and is used as the corresponding pixel change difference.

4. The sensing method for an in-vehicle camera according to claim 3, characterized in that, Also includes: The pixel change difference is calculated using multiple frames preceding the current frame image to correct the pixel change difference corresponding to the current frame image.

5. The sensing method for an in-vehicle camera according to claim 4, characterized in that, Also includes: When the vehicle's front-view camera is directly exposed to strong light, other vehicle cameras and the front-view camera simultaneously capture images of the same obstacle. In a certain frame, the pixel values ​​of the two different cameras capturing the same obstacle are compared, and the pixel change difference between the two cameras is calculated. Then, the calculated pixel change difference is used to correct the image captured after being exposed to strong light.

6. The sensing method for an in-vehicle camera according to claim 5, characterized in that, The method of simultaneously capturing images of the same obstacle using other vehicle-mounted cameras and a front-view camera includes: Use the vehicle's rearview camera and frontview camera to simultaneously capture images of the same sky, trees, railings, or road surface.

7. The sensing method for an in-vehicle camera according to claim 6, characterized in that, The calculation yields the pixel change difference between the two cameras, including: The sum of pixel values ​​V4 of N3 sky pixels in the image captured by the vehicle's front-view camera and the sum of pixel values ​​V5 of N4 sky pixels in the image detected by the rear-view camera are obtained. The average difference V6 is calculated according to the formula V6=V4 / N3-V5 / N4, and the average difference V6 is used as the corresponding pixel change difference for image correction.

8. The sensing method for an in-vehicle camera according to claim 7, characterized in that, Also includes: When the vehicle's forward-facing camera is directly exposed to strong light, the pixel change difference is calculated using different frame images of the same obstacle taken by different cameras. The calculated pixel change difference is then used to correct the image taken after being exposed to strong light.

9. The sensing method for an in-vehicle camera according to claim 8, characterized in that, The calculation of pixel change differences using different frame images of the same obstacle taken by different cameras includes: Images of the same obstacle are captured using both the vehicle's front-view camera and rear-view camera. Get the sum of pixel values ​​V7 of N5 pixels in the image of the obstacle captured by the vehicle's front-view camera, and the sum of pixel values ​​V8 of N6 pixels in the image of the obstacle captured by the vehicle's rear-view camera in any subsequent frame. The average difference V9 is calculated using the formula V9=V7 / N5-V8 / N6, and the average difference V6 is used as the corresponding pixel change difference for image correction.

10. The sensing method for an in-vehicle camera according to claim 9, characterized in that, Also includes: When an obstacle is detected in front of the vehicle, the detection information is recorded and stored, and then searched in images within a set frame range of other cameras to determine if there is a similar obstacle. If so, it is considered the same obstacle, and the pixel change difference is calculated using different frame images of the same obstacle taken by different cameras. The calculated pixel change difference is then used to correct the image taken after being illuminated by strong light.

Images