Imaging method and system for dynamically eliminating ghost images of streaming media rearview mirror

Abstract

The present invention provides an imaging method and system for dynamically eliminating ghost images of a streaming media rearview mirror. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror comprises the following steps: a vehicle-mounted camera obtains the image information of the rear of the vehicle and sends it to an image processing and display module of the vehicle-mounted streaming media rearview mirror; the image processing and display module extracts the brightness information of each pixel; the image processing and display module compares the brightness information with a preset brightness threshold, performs binarization processing to distinguish each pixel as a light source or a background; the image processing and display module calculates the number of light sources and the number of backgrounds; the image processing and display module determines whether the current image is a point light source in a dark environment; if it is determined to be a point light source in a dark environment, controls the vehicle-mounted camera to reduce the sensitivity and exposure shutter speed; and the vehicle-mounted streaming media rearview mirror displays the image information of the rear of the vehicle. The beneficial effects are: reducing the complexity of the calculation data and the requirements of the calculation power, and realizing the real-time detection of ghost images by the vehicle-mounted streaming media rearview mirror.

Description

Technical Field

[0001] The invention relates to the technical field of vehicle-mounted streaming media rearview mirror equipment, and in particular to an imaging method and system for dynamically eliminating ghost images of a streaming media rearview mirror. Background Art

[0002] With the development of the automobile industry, streaming media rearview mirrors (including interior rearview mirrors and exterior rearview mirrors) are playing an increasingly important role. Compared with traditional rearview mirrors, streaming media rearview mirrors can directly capture the image behind the vehicle through a camera, presenting a wider field of view and a more intuitive image, which can greatly reduce visual blind spots. When driving at night, the extremely strong night vision capability of the streaming media camera can play a huge role, allowing the driver to clearly observe the traffic conditions behind at night. In addition, the streaming media exterior rearview mirror can reduce wind resistance, reduce wind noise, improve aerodynamic performance, and also improve fuel economy.

[0003] In the on-board camera system of the streaming rearview mirror, ghosting is a common optical phenomenon, usually caused by reflection and scattering inside the lens. When the camera captures a high-brightness point light source (such as the headlights of oncoming vehicles, street lights, etc.), unreal double images or virtual images may be generated in the image. These virtual images are usually called "ghost images", and the scene where "ghost images" are more likely to occur is a point light source in a dark environment. The point light source in a dark environment is interpreted as the overall dark environment of the picture. When a small range of light sources appears in the dark environment, that is, there is a car behind that emits light to illuminate the on-board camera, ghost images are more likely to occur.

[0004] To solve the problem of ghost images affecting image information, the exposure parameters or dynamic range can be adjusted to effectively reduce the generation of ghost images when the vehicle-mounted camera processes high-brightness point light sources. For example, the patent "A control method, device and vehicle-mounted image capture system for a vehicle-mounted camera" (application number 201910244126.6) discloses a control method, device and vehicle-mounted image capture system for a vehicle-mounted camera, which obtains the light intensity when multiple cameras set at different positions of the vehicle capture images; determines the illumination difference between the illumination intensities at the positions when any two cameras capture images; and in response to the illumination difference exceeding the preset threshold, outputs a control signal to at least control the camera at the position with large illumination intensity to adjust the image acquisition parameters to capture the image. The present invention also provides a control device and an image capture system for implementing the above control method. According to the scheme provided by the invention, it is possible to capture images containing dark details in a backlit environment, thereby improving driving safety and ensuring accurate display of image information.

[0005] Although the existing solutions can adjust the correct picture information and reduce the impact of ghost images on the image by adjusting parameters such as shutter speed, ISO sensitivity, and aperture, they require the use of multiple cameras in coordination and calculation of the images captured by two cameras respectively. The algorithm is relatively complex and requires a more expensive computing module. There is a certain processing delay, which cannot meet the user experience and is not conducive to improving the competitiveness of the product. Summary of the invention

[0006] In order to solve the problems in the prior art, the present invention provides an imaging method and system for dynamically eliminating ghost images of a streaming media rearview mirror.

[0007] The present invention provides an imaging method for dynamically eliminating ghost images of a streaming media rearview mirror, comprising the following steps:

[0008] Step S1, the vehicle-mounted camera obtains the image information behind the vehicle;

[0009] Step S2, the vehicle camera sends the rear view information of the vehicle to the image processing and display module of the vehicle streaming media rearview mirror;

[0010] Step S3, the brightness extraction module in the image processing and display module extracts the brightness information of each pixel in the image information behind the vehicle;

[0011] Step S4, the processing module of the image processing and display module compares the brightness information of each pixel with a preset brightness threshold, and performs binarization processing to distinguish each pixel as a light source or a background;

[0012] Step S5, the statistical module of the image processing and display module counts the number of light sources and the number of background pixels in the image information behind the vehicle;

[0013] Step S6: The statistical module of the image processing and display module determines whether the current image is a point light source in a dark environment;

[0014] Step S7: if it is determined to be a point light source in a dark environment, the image processing and display module controls the vehicle-mounted camera to reduce the sensitivity and exposure shutter speed;

[0015] Step S8: The display screen of the vehicle-mounted streaming media rearview mirror displays the vehicle rear image information sent by the vehicle-mounted camera.

[0016] The present invention is further improved, and the specific steps of the binarization process in step S4 are as follows:

[0017] Step A1: Pixels with brightness higher than the threshold are marked as 1, which represents the light source, and pixels with brightness lower than the threshold are marked as 0, which represents the background;

[0018] Step A2: After the pixel-by-pixel comparison is completed, a binary image is obtained;

[0019] Step A3: Complete the binarization process.

[0020] The present invention is further improved, and the specific steps of performing statistics in step S5 are as follows:

[0021] Step B1: Divide the image information behind the vehicle into a plurality of rectangular detection areas with the same number of pixels, count the plurality of rectangular detection areas one by one, and determine the ratio between the number of light sources and the number of background pixels in the area;

[0022] Step B2: If the number of light sources in the area is greater than the number of background light sources, the area is preliminarily determined to be a point light source; if the number of light sources in the area is less than the number of background light sources, the area is preliminarily determined to be a background;

[0023] Step B3: After the statistics of multiple pairs of rectangular detection areas are completed one by one, the statistical process is completed.

[0024] The present invention is further improved. In the specific judgment process of step S6, when the number of the multiple rectangular detection areas that meet the point light source standard in step S5 is one or two, it is confirmed that the current picture is a dark environment point light source.

[0025] The present invention is further improved. In the specific judgment process in step S6, when the number of point light source standards that meet the requirements in multiple rectangular detection areas in step S5 is zero or greater than two, it is confirmed that the current picture is not a point light source in a dark environment. In step S7, if it is determined that the environment is not a point light source in a dark environment, no adjustment is made to the vehicle-mounted camera, and the rear image information of the vehicle sent by the vehicle-mounted camera is displayed on the display screen.

[0026] The present invention is further improved in that the vehicle rear picture information includes a high-weight area and two low-weight areas, and the two low-weight areas are arranged on both sides of the high-weight area. In step S5, only the high-weight area is counted.

[0027] The present invention is further improved by further including the following steps between step S3 and step S4:

[0028] Step C1: The signal filtering module in the image processing and display module obtains the brightness information of each pixel point, and performs noise filtering processing on each pixel point through the filter in the signal filtering module;

[0029] Step C2: The filter sends the filtered brightness information to the processing module in step S4.

[0030] The present invention is further improved. When the image processing and display module receives the rear image information of the vehicle, the rear image information of the vehicle is read line by line. After the image processing and display module reads the first line, steps S3, S4, S5 and S6 start to extract and judge the image data in sequence, and work in parallel to shorten the process processing time.

[0031] The present invention is further improved. After the specified time is displayed in step S8, the time control module of the image processing and display module will control the image processing and display module to jump to step S3 for execution. When it is determined in step S7 that the scene is not a point light source in a dark environment, the image processing and display module controls the exposure parameters of the vehicle-mounted camera to be restored to the initial value.

[0032] The present invention also provides a system, including a vehicle-mounted streaming media rearview mirror, a transmission cable and a vehicle-mounted camera, wherein the vehicle-mounted camera is electrically connected to the vehicle-mounted streaming media rearview mirror via the transmission cable.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides an imaging method and system for dynamically eliminating ghost images of a streaming media rearview mirror, which can effectively solve the problem of multi-camera coordination and complex calculation in the prior art for eliminating the influence of ghost images on the screen. By using this method, the brightness extraction module, the processing module and the statistical module cooperate to efficiently identify the scene of the ghost image, and the data information is processed by binarization, which reduces the complexity of the calculation data and the requirement of the calculation power, realizes the real-time detection of ghost images by the vehicle-mounted streaming media rearview mirror, quickly adjusts the exposure parameters of the vehicle-mounted camera, eliminates the influence of ghost images on the driver's understanding of the surrounding environment, improves the user experience, and is conducive to improving the competitiveness of the product. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the solutions in the present application or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A system block diagram of the present invention;

[0036] Figure 2 The present invention provides a flow chart of an imaging method for dynamically eliminating ghost images of a streaming media rearview mirror. DETAILED DESCRIPTION

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field of this application; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit the application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0038] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0040] like Figure 1 As shown, a system includes a vehicle-mounted streaming media rearview mirror and a vehicle-mounted camera, and the vehicle-mounted camera is electrically connected to the vehicle-mounted streaming media rearview mirror through a transmission cable.

[0041] The vehicle-mounted streaming media rearview mirror is provided with a controller, which is a programmable logic device (FPGA). The controller is provided with an image processing and display module. The image processing and display module is provided with a point light source statistics module and a display screen. The point light source statistics module is provided with a brightness extraction module, a processing module, a statistics module and a filtering module.

[0042] The present invention provides an imaging method for dynamically eliminating ghost images of a streaming media rearview mirror, such as Figure 2 As shown, this method is used in the above system. The detailed implementation method of the dynamic elimination of ghost images of the streaming media rearview mirror in this example is as follows:

[0043] 1. The vehicle-mounted camera obtains the image information behind the vehicle.

[0044] 2. The vehicle-mounted camera sends the rear view information of the vehicle to the image processing and display module of the vehicle-mounted streaming media rearview mirror.

[0045] The vehicle-mounted camera sends the rear-view image information of the vehicle to the image processing and display module of the vehicle-mounted streaming media rearview mirror in real time via a transmission cable.

[0046] 3. The brightness extraction module in the image processing display module extracts the brightness information of each pixel in the image information behind the vehicle.

[0047] After the image processing and display module receives the rear image information of the vehicle, the brightness extraction module in the image processing and display module reads the brightness information of the rear image information of the vehicle to facilitate the statistics of point light sources in subsequent processes.

[0048] The decompressed image signal of the vehicle camera includes YUV, RGB, or YCbCr format. The Y component in the YUV and YCbCr formats directly represents the brightness information, and U and V are the chrominance information respectively. The YUV and YCbCr formats can directly extract the brightness information.

[0049] For RGB format, pass Calculated.

[0050] The following steps are also included between step S3 and step S4:

[0051] Step C1: The signal filtering module in the image processing and display module obtains the brightness information of each pixel point, and performs noise filtering processing on each pixel point through the filter in the signal filtering module;

[0052] Step C2: The filter sends the filtered brightness information to the processing module in step S4.

[0053] By setting the signal filtering module (low-pass filtering), flickering or other electronic problems in the rear image information of the vehicle obtained by the on-board camera during driving can be suppressed, the information can be prevented from receiving external interference, the erroneous binarization results caused by electromagnetic interference or sensor noise can be reduced, and the accuracy of the image information can be improved.

[0054] The patented part uses a low-pass filter with parameters such as (0.40.20.4) to prevent sudden flashes in certain bands from causing screen flickering.

[0055] 4. The processing module of the image processing display module compares the brightness information of each pixel with the preset brightness threshold, and performs binarization processing to distinguish each pixel as a light source or background.

[0056] The specific steps for binarization are as follows:

[0057] Step A1: Pixels with brightness higher than the threshold are marked as 1, which represents the light source, and pixels with brightness lower than the threshold are marked as 0, which represents the background;

[0058] Step A2: After the pixel-by-pixel comparison is completed, a binary image is obtained;

[0059] Step A3: Complete the binarization process.

[0060] Through binarization processing, the data can be effectively simplified and the calculation pressure of the algorithm can be reduced.

[0061] 5. The statistical module of the image processing display module counts the number of light sources and the number of background pixels in the image information behind the vehicle.

[0062] The specific steps for statistical analysis are as follows:

[0063] Step B1: Divide the image information behind the vehicle into a plurality of rectangular detection areas with the same number of pixels, count the plurality of rectangular detection areas one by one, and determine the ratio between the number of light sources and the number of background pixels in the area;

[0064] Step B2: If the number of light sources in the area is greater than the number of background light sources, the area is preliminarily determined to be a point light source; if the number of light sources in the area is less than the number of background light sources, the area is preliminarily determined to be a background;

[0065] Step B3: After the statistics of multiple pairs of rectangular detection areas are completed one by one, the statistical process is completed.

[0066] Statistics are performed through rectangular detection areas to effectively improve the accuracy of identifying point light sources in dark environments. This is more in line with the situation where the light source is in a circular shape and shines on the CMOS on the vehicle-mounted camera. The pixels of a single light source and a point light source are gathered together. If the pixels of the point light source are scattered in the picture, it is judged that the current picture is not a point light source in a dark environment.

[0067] The size of the rectangular detection area can be three pixels long and three pixels wide, and the pixel area of ​​three rows and three columns is counted at the same time, or it can be five pixels long and five pixels wide, and the pixel area of ​​five rows and five columns is counted at the same time, and the scanning and statistics are performed from left to right and from top to bottom.

[0068] When a rectangular detection area with a length of five pixels and a width of five pixels is used for scanning statistics, when 13-16 pixels out of the twenty-five pixels are point light sources, that is, the number of light sources accounts for 50%-70%, the area is judged to be a point light source area.

[0069] The vehicle rear image information includes a high-weight area and two low-weight areas. The two low-weight areas are arranged on both sides of the high-weight area. In step S5, only the high-weight area is counted.

[0070] By only counting high-weight areas, the amount of data for calculation can be effectively reduced, and the low-weight areas have little effect on the appearance of ghost images and the probability is low. By counting whether dark environment point light sources appear in high-weight areas, it is possible to determine whether ghost images appear in the current picture, further reducing the processing delay of automatically eliminating or weakening ghost images, thereby improving the user experience.

[0071] 6. The statistical module of the image processing and display module determines whether the current image is a point light source in a dark environment.

[0072] In the specific judgment process of step S6, when the number of the plurality of rectangular detection areas satisfying the point light source standard in step S5 is one or two, it is determined that the current picture is a dark environment point light source.

[0073] Ghosting is more likely to occur when the point light source is small. If a large area of ​​light appears in the entire screen, ghosting is less likely to occur. Therefore, in scenes with high incidence of ghosting, point light sources are subject to size restrictions. There may only be one or two rectangular detection areas that meet the standard requirements for point light sources. Two requirements mean that the light source is divided into two rectangular detection areas.

[0074] In the specific judgment process in step S6, when the number of point light source standards that meet the requirements in multiple rectangular detection areas in step S5 is zero or greater than two, it is confirmed that the current picture is not a point light source in a dark environment. In step S7, if it is determined that the environment is not a point light source in a dark environment, no adjustment is made to the vehicle-mounted camera, and the rear image information of the vehicle sent by the vehicle-mounted camera is displayed on the display screen.

[0075] When "in the specific judgment process in step S6, when the number of point light sources that meet the standard in multiple rectangular detection areas in step S5 is zero or greater than two", that is, the picture may be in a completely dark environment or multiple light sources appear in the picture, then in this state, the probability of ghost images appearing is low, and the picture is not adjusted to ensure the accuracy of the picture display.

[0076] When the image processing and display module receives the information about the rear view of the vehicle, the information about the rear view of the vehicle is read line by line. After the image processing and display module finishes reading the first line, steps S3, S4, S5 and S6 start to extract and judge the image data in sequence, and the work is done in parallel to shorten the process processing time.

[0077] 7. If it is determined to be a point light source in a dark environment, the image processing and display module controls the on-board camera to reduce the sensitivity and exposure shutter speed.

[0078] By adjusting the sensitivity and exposure shutter speed, the impact of ghost images on the image can be effectively reduced.

[0079] 8. The display screen of the vehicle-mounted streaming media rearview mirror displays the vehicle rear view information sent by the vehicle-mounted camera.

[0080] After the on-board camera adjusts the parameters, the picture will change to a certain extent, reducing the impact of ghost images, and the information will be sent to the on-board streaming media rearview mirror. The on-board camera will continue to record the picture through the generated information to complete the process of eliminating ghost images from the picture.

[0081] After the in-vehicle streaming media displays the specified time, the time control module of the image processing and display module will control the image processing and display module to jump to the third point mentioned above and continue to execute downward. When it executes to the seventh point mentioned above and judges that the scene is not a point light source in a dark environment, the image processing and display module controls the exposure parameters of the in-vehicle camera to be restored to the initial value. After the in-vehicle camera adjusts the parameters, it will be sent to the in-vehicle streaming media rearview mirror to realize automatic adjustment of the in-vehicle camera and automatically eliminate ghost images, thereby improving the user experience.

[0082] 9. Dynamic ghost elimination is completed.

[0083] Through the above process, it is possible to effectively recognize and eliminate ghost images through a single vehicle-mounted camera in conjunction with an image processing and display module without changing or adding to the original vehicle-mounted camera hardware, thereby simplifying the calculation amount of the vehicle-mounted streaming media rearview mirror and achieving real-time ghost image recognition and processing. Traditional image recognition technologies include high dynamic range (HDR) technology / light source detection and dynamic adjustment, etc., because they are relatively complex, have a large amount of calculation, have delays, and are not suitable for vehicle-mounted system implementation. The use of this solution improves the user experience.

[0084] As can be seen from the above, the present invention provides an imaging method and system for dynamically eliminating ghost images of a streaming media rearview mirror, which can effectively solve the problem of multi-camera coordination and complex calculation in the way of eliminating the influence of ghost images on the screen in the prior art. By using this method, the brightness extraction module, the processing module and the statistical module cooperate to efficiently identify the scene of the ghost image, and the data information is processed by binarization, which reduces the complexity of the calculation data and the requirement of the calculation power, realizes the real-time detection of ghost images by the vehicle-mounted streaming media rearview mirror, quickly adjusts the exposure parameters of the vehicle-mounted camera, eliminates the influence of ghost images on the driver's understanding of the surrounding environment, improves the user experience, and is conducive to improving the competitiveness of the product.

[0085] The specific implementation modes described above are preferred implementation modes of the present invention, and are not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to the specific implementation modes, and all equivalent changes made according to the present invention are within the protection scope of the present invention.

Claims

1. An imaging method for dynamically eliminating ghost images of a streaming media rearview mirror, characterized in that: The steps include: Step S1, the vehicle-mounted camera obtains the image information behind the vehicle; Step S2, the vehicle camera sends the rear view information of the vehicle to the image processing and display module of the vehicle streaming media rearview mirror; Step S3, the brightness extraction module in the image processing and display module extracts the brightness information of each pixel in the image information behind the vehicle; Step S4, the processing module of the image processing and display module compares the brightness information of each pixel with a preset brightness threshold, and performs binarization processing to distinguish each pixel as a light source or a background; Step S5, the statistical module of the image processing and display module counts the number of light sources and the number of background pixels in the image information behind the vehicle; Step S6: The statistical module of the image processing and display module determines whether the current image is a point light source in a dark environment; Step S7: if it is determined to be a point light source in a dark environment, the image processing and display module controls the vehicle-mounted camera to reduce the sensitivity and exposure shutter speed; Step S8: The display screen of the vehicle-mounted streaming media rearview mirror displays the vehicle rear image information sent by the vehicle-mounted camera.

2. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: The specific steps of the binarization process in step S4 are as follows: Step A1: Pixels with brightness higher than the threshold are marked as 1, which represents the light source, and pixels with brightness lower than the threshold are marked as 0, which represents the background; Step A2: After the comparison of each pixel is completed, a binary image is obtained; Step A3: Complete the binarization process.

3. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: The specific steps of performing statistics in step S5 are as follows: Step B1: Divide the image information behind the vehicle into a plurality of rectangular detection areas with the same number of pixels, count the plurality of rectangular detection areas one by one, and determine the ratio between the number of light sources and the number of background pixels in the area; Step B2: If the number of light sources in the area is greater than the number of background light sources, the area is preliminarily determined to be a point light source; if the number of light sources in the area is less than the number of background light sources, the area is preliminarily determined to be a background; Step B3: After the statistics of multiple pairs of rectangular detection areas are completed one by one, the statistical process is completed.

4. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 3 is characterized in that: In the specific judgment process of step S6, when the number of the plurality of rectangular detection areas satisfying the point light source standard in step S5 is one or two, it is determined that the current picture is a dark environment point light source.

5. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 4 is characterized in that: In the specific judgment process in step S6, when the number of point light source standards that meet the requirements in multiple rectangular detection areas in step S5 is zero or greater than two, it is confirmed that the current picture is not a point light source in a dark environment. In step S7, if it is determined that the environment is not a point light source in a dark environment, no adjustment is made to the vehicle-mounted camera, and the rear image information of the vehicle sent by the vehicle-mounted camera is displayed on the display screen.

6. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: The vehicle rear image information includes a high-weight area and two low-weight areas. The two low-weight areas are arranged on both sides of the high-weight area. In step S5, only the high-weight area is counted.

7. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: The following steps are also included between step S3 and step S4: Step C1: The signal filtering module in the image processing and display module obtains the brightness information of each pixel point, and performs noise filtering processing on each pixel point through the filter in the signal filtering module; Step C2: The filter sends the filtered brightness information to the processing module in step S4.

8. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: When the image processing and display module receives the information about the rear view of the vehicle, the information about the rear view of the vehicle is read line by line. After the image processing and display module finishes reading the first line, steps S3, S4, S5 and S6 start to extract and judge the image data in sequence, and the work is done in parallel to shorten the process processing time.

9. The imaging method for dynamically eliminating ghost images of a streaming media rearview mirror according to claim 1, characterized in that: After the specified time is displayed in step S8, the time control module of the image processing and display module will control the image processing and display module to jump to step S3 for execution. When it is determined in step S7 that the scene is not a point light source in a dark environment, the image processing and display module controls the exposure parameters of the vehicle-mounted camera to be restored to the initial value.

10. A system for implementing any of the above-mentioned imaging methods for dynamically eliminating ghost images in a streaming media rearview mirror, characterized in that: It comprises a vehicle-mounted streaming media rearview mirror, a transmission cable and a vehicle-mounted camera, wherein the vehicle-mounted camera is electrically connected to the vehicle-mounted streaming media rearview mirror via the transmission cable.

Images