Intelligent automobile camera shielding real-time detection method

Abstract

The invention belongs to the technical field of automatic driving safety, and particularly relates to an intelligent automobile camera shielding real-time detection method, which comprises a plurality of cameras, an illumination image preprocessing module, a multi-scale feature extraction module and a light color histogram calculation module. And the free adaptation factor calculation module judges that the view of the camera is shielded by combining the difference of the color histograms and environmental parameters such as the relative position and angle of the camera, and then triggers corresponding warning or countermeasures. According to the real-time detection method for shielding of the intelligent automobile camera, the adaptive factor is calculated by combining the color histogram difference and various environmental parameters, and a self-learning and optimization mechanism is introduced, so that the sensitivity and the judgment threshold of shielding detection can be dynamically adjusted according to the actual situation, the accuracy and the adaptability of detection are remarkably improved, and the detection efficiency is improved. Various complex environmental conditions and illumination changes are effectively handled, and the accuracy of a detection result is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving safety, and specifically to a real-time detection method for camera occlusion of intelligent vehicles. Background Technique

[0002] With the rapid development of intelligent vehicle technology, cameras, as key environmental perception devices, play an indispensable role in realizing driving assistance functions such as lane keeping and collision warning. Intelligent vehicles rely on cameras for environmental perception to achieve driving assistance functions such as lane keeping and collision warning. However, in actual use, cameras are easily interfered by various factors, and the situation of being occluded is particularly common. Once the camera is occluded, the image information it collects will not be able to truly reflect the environmental conditions around the vehicle, which will seriously affect the reliability and accuracy of driving assistance functions, and thus pose a threat to vehicle driving safety.

[0003] Currently, although there are some methods for detecting camera occlusion, most of them have problems such as insufficient accuracy, inability to adapt to complex environmental changes, and inability to detect in real time and effectively. For example, some methods based on simple image feature analysis only rely on single features such as the brightness and color of the image to judge occlusion. When encountering scenarios such as a vehicle passing through a tunnel or being directly irradiated by strong light, since the brightness or color of the image will change violently, these methods may misjudge such normal environmental changes as the camera being occluded; and when the camera is partially occluded and the image features of the occluded part are similar to the surrounding environment, it may not be able to accurately identify the occlusion situation, resulting in missed detection, which greatly reduces the accuracy of the detection results and cannot provide reliable information for drivers and vehicle control systems. In view of this, a real-time detection method for camera occlusion of intelligent vehicles is proposed. Summary of the Invention

[0004] The main object of the present invention is to provide a real-time detection method for camera occlusion of intelligent vehicles, which can solve the problems raised in the above background technique.

[0005] To achieve the above object, the real-time detection method for camera occlusion of intelligent vehicles proposed by the present invention includes the following steps:

[0006] S1. Use multiple cameras installed on the intelligent vehicle to collect image data of the environment around the vehicle in real time;

[0007] S2. Through the illumination image preprocessing module, according to image features and environmental parameters, use the formula I pre =f light (I) to perform illumination image preprocessing on the images captured by each camera, where I is the original image, I pre is the image after illumination preprocessing, and f light is the illumination preprocessing function;

[0008] S3. With the help of the multi-scale feature extraction module, construct a Gaussian pyramid of the image, and use the formula F scale = g scale (I pre ) to extract the features of the target object at multiple scales, where F scale is the feature extracted at a specific scale, and g scale is the multi-scale analysis function;

[0009] S4. Use the ray color histogram calculation module to calculate the color histogram of the target area in each camera image. The formula is H i = h(I pre , i), where for the image of the i-th camera, H i is its color histogram, and h is the histogram calculation function;

[0010] S5. The free adaptation factor calculation module combines the differences of the color histograms and environmental parameters such as the relative positions and angles of the cameras, and uses the formula α = φ(H1, H2,..., H n ) to calculate the adaptation factor, where α is the adaptation factor, φ is the adaptation factor calculation function, and H1, H2,..., H n are the color histograms of each camera, and update the calculation model through a self-learning and optimization mechanism;

[0011] S6. Compare the adaptation factor α with the preset threshold θ through the occlusion determination threshold setting module. If α > θ, it is determined that the view of at least one camera is occluded, and corresponding warnings or countermeasures are triggered.

[0012] Preferably, the image feature preprocessing of the illumination image preprocessing module adjusts the illumination distribution of the image using histogram equalization technology. The illumination information preprocessing reduces the influence of illumination changes based on the Retinex theory, illumination balance algorithm, or histogram equalization method, and performs multi-scale analysis on the preprocessed image. The illumination image preprocessing module is responsible for preprocessing the original image collected by the camera to reduce the influence of factors such as illumination changes, non-uniform illumination, and shadows on subsequent feature extraction and analysis, and provides high-quality image data for subsequent processing.

[0013] Preferably, the image Gaussian pyramid constructed by the multi-scale feature extraction module obtains the features of the target object at different resolutions through convolution operations at different scales to adapt to the scale changes brought about by different camera perspectives and distances. To adapt to the scale changes brought about by different camera perspectives and distances, this module constructs an image Gaussian pyramid to capture the features of the target object at multiple scales to ensure that the features of the target object can be accurately extracted under different shooting conditions, providing a stable and reliable feature basis for subsequent occlusion judgment.

[0014] Preferably, when calculating the color histogram, the light color histogram calculation module statistically analyzes the distribution of the number of pixels of different colors in the image to capture the color distribution characteristics of the target. Color information is an effective feature in image processing. Especially when the lighting conditions change, the color histogram can better reflect the color distribution characteristics of the target and has a certain resistance to lighting changes. The light color histogram calculation module calculates the color histogram for the target area in each camera image, providing important data support for subsequent adaptive factor calculation.

[0015] Preferably, during the self-learning and optimization process, the adaptive factor calculation module adjusts the parameters in the adaptive factor calculation function φ according to new occlusion cases and feedback information to dynamically optimize the calculation of the adaptive factor.

[0016] Preferably, the preset threshold θ in the occlusion determination threshold setting module is preset and dynamically adjusted according to different vehicle models, usage scenarios, and safety requirements.

[0017] Preferably, when it is determined that there is a camera occlusion, the triggered countermeasures include, but are not limited to, adjusting the working modes of other sensors to make up for the information loss caused by the camera occlusion, and recording the relevant data of the occlusion event for subsequent algorithm optimization.

[0018] Preferably, when the intelligent vehicle starts, initialization settings are performed on the illumination image preprocessing module, multi-scale feature extraction module, light color histogram calculation module, adaptive factor calculation module, and occlusion determination threshold setting module.

[0019] Preferably, the illumination preprocessing function f in the illumination image preprocessing module light dynamically determines the preprocessing operations according to parameters such as the brightness, contrast, texture of the image, as well as the ambient light intensity and weather conditions.

[0020] Preferably, the adaptive factor calculation function φ in the adaptive factor calculation module comprehensively considers the similarities, differences, and deviations from the reference histogram among the color histograms of each camera to calculate an adaptive factor that accurately reflects the occlusion possibility.

[0021] The present invention provides a method for real-time detection of camera occlusion in intelligent vehicles. It has the following beneficial effects:

[0022] (1) By combining the color histogram difference and various environmental parameters to calculate the adaptive factor and introducing a self-learning and optimization mechanism, this method for real-time detection of camera occlusion in intelligent vehicles can dynamically adjust the sensitivity and determination threshold of occlusion detection according to the actual situation, significantly improving the accuracy and adaptability of detection, effectively coping with various complex environmental conditions and light changes, and effectively improving the accuracy of detection results.

[0023] (2) By comprehensively using image illumination features, multi-scale target object features, and color histogram features for occlusion detection, this method for real-time detection of camera occlusion in intelligent vehicles fully exploits various information in the image. Compared with traditional single-feature detection methods, the detection results are more accurate and reliable.

[0024] (3) By realizing real-time monitoring and rapid response to the camera state, and at the same time the system can self-optimize according to past experience, continuously adjust the algorithm parameters, improve the stability and reliability of long-term operation, and adapt to the complex and changeable usage scenarios of intelligent vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0026] Figure 1 is the flowchart of the method for real-time detection of camera occlusion in intelligent vehicles of the present invention;

[0027] Figure 2 is the flowchart of the illumination image preprocessing module of the present invention;

[0028] Figure 3 is the flowchart of the multi-scale feature extraction module of the present invention;

[0029] Figure 4 is the flowchart of the light color histogram calculation module of the present invention;

[0030] Figure 5 is the flowchart of the adaptive factor calculation module of the present invention;

[0031] Figure 6 is the flowchart of the occlusion determination threshold setting module of the present invention.

[0032] The realization, functional features, and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed implementation manners

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] Please refer to Figures 1 - 6 , the present invention proposes a method for real-time detection of intelligent vehicle camera occlusion, including the following steps:

[0035] S1. Use multiple cameras installed on the intelligent vehicle to collect image data of the vehicle surrounding environment in real time;

[0036] S2. Through the illumination image preprocessing module, according to the image features and environmental parameters, use the formula I pre = f light (I) to perform illumination image preprocessing on the images captured by each camera, where I is the original image, I pre is the image after illumination preprocessing, and f light is the illumination preprocessing function;

[0037] Among them, the image feature preprocessing of the illumination image preprocessing module uses histogram equalization technology to adjust the illumination distribution of the image, and the illumination information preprocessing reduces the influence of illumination changes based on the Retinex theory, illumination balance algorithm, or histogram equalization method, and performs multi-scale analysis on the preprocessed image;

[0038] S3. With the help of the multi-scale feature extraction module, construct an image Gaussian pyramid, and use the formula F scale = g scale (I pre ) to extract the features of the target object at multiple scales, where F scale is the feature extracted at a specific scale, and g scale is the multi-scale analysis function;

[0039] Among them, the image Gaussian pyramid constructed by the multi-scale feature extraction module obtains the features of the target object at different resolutions through convolution operations at different scales to adapt to the scale changes brought by different camera perspectives and distances;

[0040] S4. Use the ray color histogram calculation module to calculate the color histogram of the target area in each camera image, and the formula is H i = h(I pre, i), where for the image of the i-th camera, H i is its color histogram, and h is the histogram calculation function;

[0041] Among them, when the light color histogram calculation module calculates the color histogram, it counts the pixel quantity distribution of different colors in the image to capture the color distribution characteristics of the target;

[0042] S5. The free adaptation factor calculation module combines the differences in color histograms and environmental parameters such as the relative positions and angles of the cameras, and uses the formula α = φ(H1, H2,..., H n ) to calculate the adaptation factor, where α is the adaptation factor, φ is the adaptation factor calculation function, and H1, H2,..., H n are the color histograms of each camera, and updates the calculation model through a self-learning and optimization mechanism;

[0043] Among them, during the self-learning and optimization process, the adaptation factor calculation module adjusts the parameters in the adaptation factor calculation function φ according to new occlusion cases and feedback information to dynamically optimize the calculation of the adaptation factor;

[0044] S6. The occlusion determination threshold setting module compares the adaptation factor α with a preset threshold θ. If α > θ, it is determined that the view of at least one camera is occluded, and corresponding warnings or countermeasures are triggered;

[0045] Among them, the preset threshold θ in the occlusion determination threshold setting module is preset and dynamically adjusted according to different vehicle models, usage scenarios, and safety requirements.

[0046] In step S2, the image feature preprocessing of the light image preprocessing module uses histogram equalization technology to adjust the light distribution of the image. The light information preprocessing reduces the influence of light changes based on the Retinex theory, light balance algorithm or histogram equalization method, and performs multi-scale analysis on the preprocessed image to provide high-quality image data for subsequent processing. Among them, the image feature preprocessing dynamically adjusts the image preprocessing process according to the features of the image itself (such as brightness, contrast, texture, etc.) and environmental parameters (such as light intensity, weather conditions, etc.). Using histogram equalization technology, the images captured by each camera are processed. Histogram equalization can optimize the gray-level distribution of the image, enhance the contrast of the image, and thus improve the recognition rate of the target object in the image. For example, in a darker environment, the originally blurred image can be made clearer through histogram equalization, highlighting the features of the target object. The light information preprocessing can effectively remove the influence of uneven light while retaining the details of the image. The light balance algorithm can make the light in different regions more uniform by adjusting the overall light level of the image. After that, multi-scale analysis is performed on the preprocessed image. Through transformations at different scales, the light features of the image at different resolutions are extracted to more comprehensively capture the light information of the image.

[0047] In step S3, the multi-scale feature extraction module can obtain the feature information of the target object at different scales by extracting features on different layers. It performs operations such as convolution on the image at different scales according to the characteristics and requirements of the image, so as to obtain the features of the target object at different resolutions. For example, the overall contour features of the target object can be extracted at a larger scale, while the detailed features of the target object can be extracted at a smaller scale.

[0048] The function in step S4 counts the distribution of different colors according to the pixel values of the image. For example, for an RGB image, the color histograms of the red, green, and blue channels can be respectively counted. The histogram of each channel can divide the color range into several intervals, and count the number of pixels in each interval to obtain the color distribution of that channel. By analyzing and comparing the color histograms of multiple camera images, the differences in color distribution between the images can be found, and then it can be judged whether there is an occlusion situation.

[0049] The function in step S5 comprehensively considers the mutual relationship between the color histograms of multiple cameras and their differences from the reference histogram. For example, the similarity or difference degree between the color histograms of different cameras can be calculated, and then combined with the installation position and angle information of the cameras to determine the value of the adaptive factor. At the same time, this module will continuously learn new data, optimize the calculation model of the adaptive factor according to new occlusion cases and feedback information, and adjust the judgment threshold and parameters to adapt to different environments and scenarios.

[0050] In step S6, the preset threshold θ can be flexibly adjusted according to different vehicle models, usage scenarios, and safety requirements. In practical applications, a large number of experiments and tests are required to determine the appropriate preset threshold. For example, in scenarios with high safety requirements, the preset threshold can be appropriately reduced to improve the detection sensitivity; while in some scenarios that are more sensitive to false alarms, the preset threshold can be appropriately increased to reduce the situation of misjudgment. When the calculated adaptive factor is greater than the preset threshold, the system will determine that there is a camera occlusion situation, and notify the driver by displaying a prompt message on the in-vehicle display screen, emitting an alarm sound, etc., and at the same time, corresponding measures such as adjusting the working mode of other sensors can be taken to make up for the information loss caused by the camera occlusion.

[0051] In the embodiment of the present invention, when the intelligent vehicle starts, initialization settings are performed on the light image preprocessing module, multi-scale feature extraction module, light color histogram calculation module, adaptive factor calculation module, and occlusion determination threshold setting module. The light preprocessing function f in the light image preprocessing module light Dynamically determines the preprocessing operation according to parameters such as the brightness, contrast, texture of the image, as well as the ambient light intensity and weather conditions. The adaptive factor calculation function φ in the adaptive factor calculation module comprehensively considers the similarities, differences between the color histograms of each camera, and the deviation from the reference histogram to calculate an adaptive factor that accurately reflects the possibility of occlusion. In this way, by combining the color histogram difference and various environmental parameters to calculate the adaptive factor and introducing a self-learning and optimization mechanism, it is possible to dynamically adjust the sensitivity and determination threshold of occlusion detection according to the actual situation, significantly improve the detection accuracy and adaptability, effectively cope with various complex environmental conditions and light changes, and effectively improve the accuracy of the detection result.

[0052] At the same time, by comprehensively using image light characteristics, multi-scale target object characteristics, and color histogram characteristics for occlusion detection, fully excavating various information in the image, compared with the traditional single-feature detection method, the detection result is more accurate and reliable, and real-time monitoring and rapid response to the camera state are achieved. The system can self-optimize according to past experience, continuously adjust the algorithm parameters, improve the stability and reliability of long-term operation, and adapt to the complex and changeable usage scenarios of intelligent vehicles.

[0053] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural transformation made under the inventive concept of the present invention, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. A real-time detection method for occlusion of an intelligent vehicle camera, characterized in that, Including the following steps: S1. Use multiple cameras installed on the intelligent vehicle to collect image data of the vehicle surrounding environment in real time; S2. Through the illumination image preprocessing module, according to the image features and environmental parameters, using the formula I pre = f light (I) Perform illumination image preprocessing on the images captured by each camera, where I is the original image, and I pre is the image after illumination preprocessing, and f light is the illumination preprocessing function; S3. With the help of the multi-scale feature extraction module, construct an image Gaussian pyramid, and use the formula F scale = g scale (I pre ) to extract the features of the target object at multiple scales, where F scale is the feature extracted at a specific scale, and g scale is the multi-scale analysis function; S4. Use the light color histogram calculation module to calculate the color histogram of the target area in each camera image. The formula is H i = h(I pre , i), where for the image of the i-th camera, H i is its color histogram, and h is the histogram calculation function; S5. The free adaptation factor calculation module combines the differences in color histograms and environmental parameters such as the relative position and angle of the camera, and uses the formula α = φ(H1, H2,..., H n ) to calculate the adaptation factor, where α is the adaptation factor, φ is the adaptation factor calculation function, and H1, H2,..., H n are the color histograms of each camera, and updates the calculation model through a self-learning and optimization mechanism; S6. Compare the adaptive factor α with the preset threshold θ through the occlusion determination threshold setting module. If α>θ, it is determined that the view of at least one camera is occluded, and corresponding warnings or countermeasures are triggered.

2. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, wherein, The image feature preprocessing of the illumination image preprocessing module adjusts the illumination distribution of the image by using histogram equalization technology. The illumination information preprocessing reduces the influence of illumination changes based on the Retinex theory, illumination balance algorithm or histogram equalization method, and performs multi-scale analysis on the preprocessed image.

3. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, wherein, The image Gaussian pyramid constructed by the multi-scale feature extraction module obtains the features of the target object at different resolutions through convolution operations at different scales to adapt to the scale changes brought by different camera perspectives and distances.

4. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, wherein, When calculating the color histogram, the light color histogram calculation module counts the pixel quantity distribution of different colors in the image to capture the color distribution characteristics of the target.

5. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, wherein, In the self-learning and optimization process, the adaptive factor calculation module in the adaptive factor calculation module adjusts the parameters in the adaptive factor calculation function φ according to new occlusion cases and feedback information to dynamically optimize the calculation of the adaptive factor.

6. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, wherein, The preset threshold θ in the occlusion determination threshold setting module is preset and dynamically adjusted according to different vehicle models, usage scenarios and safety requirements.

7. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, characterized in that When it is determined that there is camera occlusion, the triggered countermeasures include, but are not limited to, adjusting the working modes of other sensors to make up for the information loss caused by camera occlusion, and recording the relevant data of the occlusion event for subsequent algorithm optimization.

8. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, characterized in that, When the intelligent vehicle starts, initialize the illumination image preprocessing module, multi-scale feature extraction module, light color histogram calculation module, adaptive factor calculation module and occlusion determination threshold setting module.

9. A real-time detection method for occlusion of an intelligent vehicle camera according to claim 1, characterized in that, The illumination preprocessing function f in the illumination image preprocessing module light Dynamically determines the preprocessing operations according to parameters such as the brightness, contrast, texture of the image, as well as the environmental illumination intensity and weather conditions.

10. The real-time detection method for the occlusion of an intelligent vehicle camera according to claim 1, characterized in that, The adaptive factor calculation function φ in the adaptive factor calculation module comprehensively considers the similarities, differences between the color histograms of each camera and the deviation from the reference histogram to calculate an adaptive factor that accurately reflects the occlusion possibility.

Images