Electronic rearview mirror dual point light source image recognition optimization method and system

Abstract

An embodiment of the present invention provides an optimization method and system for image recognition of dual-point light sources of an electronic rearview mirror, belonging to the technical field of dual-point light source testing. The method includes: collecting environmental information of the current driving scenario, and matching a corresponding optimization scheme for image recognition of dual-point light sources based on the environmental information; adjusting the camera parameters of the electronic rearview mirror based on the matched optimization scheme for image recognition of dual-point light sources, and performing image acquisition based on the camera after parameter adjustment to obtain an image to be recognized; performing optimization processing on the image to be recognized based on the matched optimization scheme for image recognition of dual-point light sources to obtain an optimized image; pushing the optimized image to the in-vehicle display module to perform real-time display of the optimized image. The solution of the present invention solves the problem that the existing imaging scheme of dual-point light sources cannot provide the optimal imaging effect based on the real-time driving scenario.

Description

Technical Field

[0001] The present invention relates to the technical field of dual-point light source testing, and specifically relates to an optimization method for dual-point light source image recognition of an electronic rearview mirror and an optimization system for dual-point light source image recognition of an electronic rearview mirror. Background Art

[0002] Dual-point light source imaging test is an optical test method, usually used to evaluate the imaging performance and resolution ability of an optical system. In the dual-point light source imaging test, two very close point light sources are placed in front of the optical system, and the imaging situation on the imaging plane is observed. By analyzing the positions and shapes of these two point light sources on the imaging plane, parameters such as the imaging quality, resolution ability, and distortion situation of the optical system can be evaluated.

[0003] In the existing solutions, a test environment compliant with GB15084-2022 is built to perform tuning and debugging to make the image effect meet the regulatory requirements. However, currently, the cost of building and debugging the environment is high. For general companies, the brightness of the dual-point light source image is judged based on subjective debugging to see if it is within the standard range of GB15084-2022. This is very inconvenient for the early R & D stage, and the subjective evaluation method of the debugging engineer is too single. The CMS electronic rearview mirror system refers to the "Camera Monitoring System" electronic rearview mirror system. This system uses a camera to replace the traditional rearview mirror, captures real-time images around the vehicle through the camera, and displays these images on the in-vehicle display screen to provide the driver with a clearer and more comprehensive view. During the use of the vehicle, especially in the night scene, when identifying oncoming vehicles and following vehicles, the corresponding headlight target recognition is actually the imaging of the corresponding dual-point light source. The existing dual-point light source testing and optimization algorithms cannot optimize the best imaging effect based on the actual scene changes and cannot provide the user with the optimal dual-point imaging. To solve this problem, a new optimization scheme for dual-point light source image recognition of an electronic rearview mirror needs to be proposed. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide an optimization method and system for dual-point light source image recognition of an electronic rearview mirror, so as to at least solve the problem that the existing dual-point light source image recognition solution cannot achieve the best imaging effect based on the real-time environmental conditions.

[0005] To achieve the above object, a first aspect of the present invention provides an optimization method for dual-point light source image recognition of an electronic rearview mirror. The method includes: collecting environmental information of the current driving scene, and matching a corresponding dual-point light source image recognition optimization scheme based on the environmental information; based on the matched dual-point light source image recognition optimization scheme, adjusting the camera parameters of the electronic rearview mirror, and performing image collection based on the camera after parameter adjustment to obtain an image to be recognized; based on the matched dual-point light source image recognition optimization scheme, performing optimization processing on the image to be recognized to obtain an optimized image; pushing the optimized image to the in-vehicle display module to perform real-time display of the optimized image.

[0006] Optionally, the matching of the corresponding dual-point light source image recognition optimization scheme based on the environmental information includes: generating a binary array based on the light intensity and temperature information indicated by the environmental information; performing a matching search for the dual-point light source image recognition optimization scheme in a pre-constructed scheme library based on the binary data to obtain the dual-point light source image recognition optimization scheme with the closest similarity.

[0007] Optionally, the method further includes: constructing the scheme library, including: performing simulation adaptive adjustment of the light intensity and temperature information based on the laboratory scene, and constructing a first array sample set by constructing a binary array corresponding to the light intensity and temperature information for each adjustment; collecting the light intensity and temperature information of the corresponding driving scene based on each data communication vehicle, and constructing a second array sample set by constructing a binary array corresponding to the light intensity and temperature information for each data collection; integrating the first array sample set and the second array sample set to obtain an integrated array sample set; constructing a training sample set based on the dual-point light source images corresponding to the binary arrays in the array sample set; performing light point recognition processing on each dual-point light source image based on the training sample, and determining the spot halo size of each dual-point light source image based on the recognition result; determining the camera parameter optimization scheme for the corresponding scene based on the spot halo size of each dual-point light source image, and constructing the scheme library based on all the results.

[0008] Optionally, the performing light point recognition processing on each dual-point light source image based on the training sample and determining the light source halo size of each dual-point light source image based on the recognition result includes: performing grayscale processing on the dual-point light source image to obtain a grayscale image; performing smoothing filtering processing on the grayscale image, and performing threshold segmentation processing on the grayscale image after smoothing filtering processing to obtain a segmented image; performing spot area marking on the segmented image to output a spot marked image; performing halo area recognition in the spot marked image and calculating the halo area.

[0009] Optionally, the marking of the speckle regions for the segmented image and outputting the speckle-marked image includes: performing erosion and / or dilation processing on the segmented image to obtain the speckle-marked regions; in the speckle-marked regions, performing connected component analysis, and based on the analysis result, performing speckle marking on the positions of the two light sources in the double-point light source image to output the speckle-marked image; the identifying the halo region in the speckle-marked image and calculating the area of the halo based on the area includes: performing secondary threshold segmentation on the speckle-marked image to obtain the halo region; determining the single-pixel area scale based on the light source size of the double-point light source; counting the number of pixel points in the halo region, and calculating the area of the halo region based on the number of pixel points and the single-pixel area scale.

[0010] Optionally, the optimization scheme for double-point light source image recognition includes: a camera parameter optimization scheme and an image optimization scheme; wherein, the camera parameter optimization scheme and the image optimization scheme are respectively an exposure time optimization scheme and a camera gain optimization scheme; the camera gain optimization scheme includes: Gamma value optimization, 1tm optimization, and Sharpness optimization.

[0011] Optionally, based on the optimization scheme for double-point light source image recognition by matching, adjusting the camera parameters of the electronic rearview mirror, and performing image acquisition based on the camera after parameter adjustment to obtain the image to be recognized, includes: adjusting the exposure parameter of the camera of the electronic rearview mirror based on the exposure time optimization scheme; based on the camera after adjusting the exposure parameter, continuously acquiring scene images during the vehicle driving process, and when a light source target appears in the acquired images, performing video frame capture processing, and using the frame image containing the light source target as the image to be recognized.

[0012] Optionally, based on the optimization scheme for double-point light source image recognition by matching, performing optimization processing on the image to be recognized to obtain the optimized image, includes: sequentially performing Gamma value optimization, ltm optimization, and Sharpness optimization on the image to be recognized, and real-time displaying the optimized image based on the in-vehicle display module.

[0013] The second aspect of the present invention provides an optimization system for double-point light source image recognition of an electronic rearview mirror, the system includes: an acquisition unit, configured to acquire the environmental information of the current driving scene, and match the corresponding optimization scheme for double-point light source image recognition based on the environmental information; an execution unit, configured to adjust the camera parameters of the electronic rearview mirror based on the matched optimization scheme for double-point light source image recognition, and perform image acquisition based on the camera after parameter adjustment to obtain the image to be recognized; an optimization unit, configured to perform optimization processing on the image to be recognized based on the matched optimization scheme for double-point light source image recognition to obtain the optimized image; a pushing unit, configured to push the optimized image to the in-vehicle display module to perform real-time display of the optimized image.

[0014] On the other hand, the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions run on a computer, the computer is caused to execute the above-mentioned electronic rearview mirror dual-point light source image recognition optimization method.

[0015] Through the above technical solutions, the present invention has the following beneficial effects:

[0016] 1) Environmental information matching optimization solution: By collecting the environmental information of the current driving scenario and matching the corresponding dual-point light source image recognition optimization solution, the image recognition can be optimized according to the actual environmental conditions, improving the recognition accuracy and efficiency.

[0017] 2) Camera parameter adjustment: Based on the matched optimization solution, the camera parameters of the electronic rearview mirror are adjusted, enabling the camera to adapt to different environmental conditions and providing clearer and more accurate image acquisition.

[0018] 3) Image optimization processing: Performing optimization processing on the image to be recognized can improve the image quality, reduce noise and interference, thereby enhancing the accuracy and reliability of image recognition.

[0019] 4) Real-time display of optimized images: Pushing the optimized images to the in-vehicle display module in real time allows the driver to observe the optimized images in the vehicle in real time, enhancing the driving experience and safety.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific embodiments section. Brief Description of the Drawings

[0021] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0022] Figure 1 is a flowchart of the steps of the electronic rearview mirror dual-point light source image recognition optimization method provided by an embodiment of the present invention;

[0023] Figure 2 is a system structure diagram of the electronic rearview mirror dual-point light source image recognition system provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0024] The following will describe in detail the specific embodiments of the present invention with reference to the drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not used to limit the present invention.

[0025] Figure 1It is the flowchart of the method for optimizing the image recognition of the double-point light source of the electronic rearview mirror provided by an embodiment of the present invention. As Figure 1 shown, an embodiment of the present invention provides a method for optimizing the image recognition of the double-point light source of the electronic rearview mirror. The method includes:

[0026] Step S10: Collect the environmental information of the current driving scenario, and match the corresponding optimization scheme for the image recognition of the double-point light source based on the environmental information.

[0027] Specifically, in the field of modern automotive technology, collecting the environmental information of the current driving scenario and matching the optimization scheme for the image recognition of the double-point light source based on this information is a forward-looking and innovative technological development. Through this scheme, a more intelligent and accurate electronic rearview mirror system can be realized, providing a safer and more convenient driving experience for the driver. First of all, collecting the environmental information of the current driving scenario is a crucial step. By using various sensors and cameras, the system can obtain the environmental information around the vehicle in real time, including road conditions, traffic conditions, light intensity, etc. These data can provide an accurate basis for subsequent image recognition and processing, enabling the system to make corresponding adjustments and optimizations according to the actual situation.

[0028] Matching the corresponding optimization scheme for the image recognition of the double-point light source based on the environmental information is the key step for personalized optimization of the image recognition requirements in different environments. Through the analysis and matching of the environmental information, the system can select the most suitable image recognition optimization scheme for the current scenario, such as adjusting parameters such as image brightness, contrast, and color saturation, to ensure clear and accurate image recognition results under various lighting conditions.

[0029] The beneficial effects of this scheme include but are not limited to the following points:

[0030] 1) Improve the accuracy of image recognition: By matching the optimization scheme according to the real-time environmental information, the accuracy and stability of image recognition can be effectively improved, reducing the situations of misrecognition and missed recognition, thereby enhancing the driver's perception ability of the surrounding environment.

[0031] 2) Enhance driving safety: The optimized image recognition scheme can help the driver see the situation around the vehicle more clearly, timely discover potential risk factors, and take actions in advance, thereby improving driving safety and reducing the occurrence of traffic accidents.

[0032] 3) Improve driving comfort: Precise image recognition and optimization processing can make the driver more relaxed and comfortable during driving, reducing visual fatigue and stress, and enhancing the driving experience.

[0033] 4) Intelligent driving assistance: The optimization solution based on environmental information matching endows the electronic rearview mirror system with intelligent driving assistance functions, enabling drivers to better cope with complex traffic environments and improving driving efficiency and convenience.

[0034] Preferably, the dual-point light source image recognition optimization solution corresponding to the environmental information matching includes: generating a binary array based on the light intensity and temperature information indicated by the environmental information; performing a dual-point light source image recognition optimization solution matching search in a pre-constructed solution library based on the binary data to obtain the dual-point light source image recognition optimization solution with the closest similarity.

[0035] In the embodiment of the present invention, the light and temperature of the environment will affect the performance and imaging effect of the optical system. Based on this feature, when performing dual-point light source image recognition optimization, the influencing factors of these two solutions need to be fully considered to facilitate the execution of the best imaging optimization solution for the current scenario. Based on this, the solution of the present invention first collects the light intensity and temperature information of the current scene, and then generates a corresponding binary array based on the collected information to facilitate the execution of subsequent optimal solution matching based on this binary array.

[0036] Further, the method further includes: constructing the solution library, including: performing simulation adaptive adjustment of light intensity and temperature information based on the laboratory scenario, and constructing a first array sample set by constructing a binary array corresponding to the light intensity and temperature information for each adjustment; collecting the light intensity and temperature information of the corresponding driving scenario by each data communication vehicle, and constructing a second array sample set by constructing a binary array corresponding to the light intensity and temperature information for each data collection; integrating the first array sample set and the second array sample set to obtain an integrated array sample set; constructing a training sample set based on the dual-point light source images corresponding to the binary arrays in the array sample set; performing light spot recognition processing on each dual-point light source image based on the training sample, and determining the spot halo size of each dual-point light source image based on the recognition result; determining the camera parameter optimization solution for the corresponding scenario based on the spot halo size of each dual-point light source image, and constructing a solution library based on all results.

[0037] In the embodiments of the present invention, for the integrity of the solution library, the solution of the present invention collects sample data from two aspects, namely the laboratory scenario and the real scenario. In the laboratory scenario, a simulated double-point light source is constructed, and then the light intensity and temperature information of the scenario are adaptively adjusted, and a target double-point light source image is collected once after each adjustment. In the actual scenario, the real-scenario images are collected through the in-vehicle terminals connected to each server, and then the vehicle headlight images are marked in the collected real-scenario images. The marked images are used as sample images, and the light intensity information and temperature information at the image sampling moment are determined synchronously. A complete solution library is constructed through the training samples obtained from the laboratory scenario and the training samples based on the real scenario. Further, the solution library can also be continuously updated based on new scenario images in the future.

[0038] Further, based on the training samples, perform light point recognition processing on each double-point light source image, and determine the light source halo size of each double-point light source image based on the recognition result, including: performing grayscale processing on the double-point light source image to obtain a grayscale image; performing smoothing filtering processing on the grayscale image, and performing threshold segmentation processing on the grayscale image after smoothing filtering processing to obtain a segmented image; performing spot area marking on the segmented image to output a spot-marked image; performing halo area recognition in the spot-marked image and calculating the halo area.

[0039] In the embodiments of the present invention, it specifically includes the following steps:

[0040] 1) Perform grayscale processing on the image: The grayscale processing of the image is the process of converting a color image into a grayscale image, that is, converting the RGB (red, green, blue) channel values of each pixel into a single grayscale value. This processing can simplify the image processing process, reduce the calculation amount, and retain the main features of the image. First, obtain the color image data from the input source (such as a camera, a file, etc.). A color image usually consists of three channels: red, green, and blue. For each pixel, the values of its RGB channels are weighted and averaged according to certain weights to obtain a grayscale value. The corresponding grayscale formula is: Gray = 0.299 * R + 0.587 * G + 0.114 * B. These weights are determined according to the sensitivity of the human eye to different colors. The grayscale value is usually between 0 and 255, representing different grayscale levels from black to white. After grayscale processing, it may be necessary to adjust the range of the grayscale value to ensure that the grayscale values of all pixels are within a suitable range. Recombine the grayscale values of each pixel into a grayscale image. A grayscale image has only one channel, and each pixel has only one grayscale value. Therefore, compared with a color image, a grayscale image occupies less storage space. Finally, the grayscale image can be displayed on the screen or saved as a file for subsequent processing or analysis. Through grayscale processing, the complexity of image processing can be simplified, the calculation amount can be reduced, and the main features of the image can be retained.

[0041] 2) Image smoothing filtering processing: Image smoothing filtering is a commonly used image processing technique for reducing noise in an image and smoothing the details of the image. Smoothing filtering can help improve the image quality, making the image easier to process and analyze. First, use the grayscale-processed image as the image data to be smoothed filtered. Select an appropriate smoothing filter. Commonly used smoothing filters include the mean filter, Gaussian filter, median filter, etc. Different filters have different smoothing effects and applicable scenarios. Apply the selected filter to each pixel of the image. The filter is usually a small matrix, and the smoothing effect is achieved by sliding the matrix over the image and performing weighted averaging on the surrounding pixels. When sliding the filter, the processing of boundary pixels needs to be considered. Use different boundary processing methods, such as padding boundary pixels, ignoring boundary pixels, or using specific boundary processing algorithms. Adjust the parameters of the filter, such as the size and weight of the filter, to achieve the best smoothing effect. Recombine the filtered pixels into a smoothed image. The smoothed image usually has less noise and details and is more suitable for subsequent processing or analysis. Finally, save the smoothed image as a file for subsequent processing or analysis. Through image smoothing filtering processing, noise in the image can be effectively reduced, the details of the image can be smoothed, and the image quality can be improved.

[0042] 3) Image threshold segmentation processing: First, use the image after smoothing filtering processing as the image data to be threshold segmented. According to the characteristics and requirements of the image, select an appropriate threshold. The selection of the threshold has a great impact on the segmentation result, and the optimal threshold can be determined by methods such as histogram analysis and experimental method. Apply the selected threshold to each pixel of the image, and divide the pixels into two categories: above the threshold and below the threshold. This step can obtain a binary image that only contains two pixel values. Perform post-processing on the segmented image, such as removing small regions, filling holes, connecting adjacent regions, etc., to obtain a more accurate segmentation result and obtain the spot segmentation result. Finally, the segmented image can be displayed on the screen or saved as a file for subsequent processing or analysis.

[0043] 4) Perform spot area marking: In the image after threshold segmentation, the spot area is the segmented area with specific characteristics. By analyzing the segmented image, find the position and shape of the spot area. Draw a bounding box or contour around the spot area to highlight the position and shape of the spot area, which can be achieved by drawing rectangles, polygons, or other shapes on the image. To more clearly mark the spot area, add labels or annotations near the spot area to describe the content of the target.

[0044] Further, the marking of the speckle regions in the segmented image and the output of the speckle-marked image include: performing erosion and / or dilation processing on the segmented image to obtain the speckle-marked regions; in the speckle-marked regions, performing connected component analysis, and based on the analysis result, performing speckle marking on the positions of the two light sources in the dual-point light source image, and outputting the speckle-marked image; the identifying the halo region in the speckle-marked image and calculating the area of the halo based on the area includes: performing secondary threshold segmentation on the speckle-marked image to obtain the halo region; determining the single-pixel area scale based on the light source size of the dual-point light source; counting the number of pixel points in the halo region, and calculating the area of the halo region based on the number of pixel points and the single-pixel area scale.

[0045] In the embodiments of the present invention, in the obtained marked image, the halo of the speckle region is often very large, making the distance between the two optoelectronics very small, and reducing the discrimination effect of the dual-point light source. To solve this situation, the solution of the present invention needs to identify the size of the halo, and perform threshold segmentation processing again to segment the speckle center region and the halo region. Since the area of the speckle center region is known (related to the size of the light source), the area scale of a single pixel is determined by the known area of the speckle center, and then the size of the halo can be determined based on the number of pixel points included in the segmented halo region.

[0046] Step S20: Based on the recognition optimization scheme of the matched dual-point light source image, adjust the camera parameters of the electronic rearview mirror, and perform image acquisition based on the camera after the parameter adjustment to obtain the image to be recognized.

[0047] Specifically, a halo refers to the blurred or bright effect that appears around the bright region in an image or a photo. A long exposure time means that the lens receives more light. Especially in the case of backlighting or strong light sources, a long exposure time will increase the possibility of the halo effect. A long exposure time will cause the light to be reflected and scattered multiple times inside the lens, aggravating the halo effect and making the halo more obvious. A short exposure time means that the lens receives less light, reducing the possibility of the halo effect. A short exposure time can reduce the number of reflections of light inside the lens, alleviating the halo effect and making the halo less obvious. However, an overly short exposure time will cause the camera's photosensitive element to receive insufficient light, making the image too dark and the details unable to be clearly shown. Therefore, to have less influence while ensuring that the acquired image has the ability to identify the characteristics of the speckle region, it is necessary to adjust the exposure time to an appropriate value for the outer halo. Based on this, the point light source image recognition optimization scheme of the present invention includes: a camera parameter optimization scheme and an image optimization scheme; the camera parameter optimization scheme is an exposure time optimization scheme.

[0048] In a possible implementation, for a CMS camera, it is generally 50fps or 60fps, so the maximum exposure time is 20ms or 16.6ms. Appropriately controlling the exposure time is beneficial to reducing the size of the halo.

[0049] Step S30: Based on the optimized scheme for recognizing the matched double-point light source image, perform optimization processing on the image to be recognized to obtain an optimized image.

[0050] Specifically, in addition to improving the halo by shortening the exposure time during image acquisition, the solution of the present invention will also adjust the gain during the image post-processing to improve the halo. The halo effect is reduced by adjusting parameters such as contrast and brightness.

[0051] Furthermore, Sharpness optimization refers to enhancing the clarity and details of an image through a series of techniques and methods, making the image look clearer and sharper. The clarity of an image refers to the sharpness of the edges and details of the objects in the image, which affects the visual quality and recognition ability of the image. The solution of the present invention executes the Sharpness optimization method through the following steps:

[0052] 1) Sharpening Filters: By applying a sharpening filter (such as a Laplacian filter) to enhance the edges and details of the image, highlighting the subtle changes in the image, thereby improving the clarity of the image.

[0053] 2) Local Contrast Enhancement: By enhancing the contrast of local regions of the image, the details in the image can be made more prominent, thereby improving the clarity and visual effect of the image.

[0054] 3) Super-Resolution Reconstruction: By using super-resolution technology to reconstruct a low-resolution image into a high-resolution image, thereby enhancing the clarity and details of the image.

[0055] 4) Deblurring: Removing the blurring effect in the image, restoring the clarity and details of the image, making the image look clearer and sharper.

[0056] 5) Local Detail Enhancement: Performing detail enhancement on specific regions or features in the image, highlighting the important details in the image, and improving the clarity and quality of the image.

[0057] In the embodiment of the present invention, by applying the above Sharpness optimization method, the visual quality of the image can be improved, making the image clearer and sharper.

[0058] Step S40: Push the optimized image to the in-vehicle display module to perform real-time display of the optimized image.

[0059] Specifically, after selecting the corresponding optimization scheme based on the environmental information of the current scene, the video stream information collected is optimized in real time, and then the optimization result is displayed in real time, so that the display module on the in-vehicle side always displays the most optimized dual-point light source imaging image, ensuring the user experience and driving safety.

[0060] Figure 2 It is the system structure diagram of the dual-point light source image recognition and optimization system of the electronic rearview mirror provided by an embodiment of the present invention. As Figure 2 shown, an embodiment of the present invention provides a dual-point light source image recognition and optimization system for an electronic rearview mirror, and the system includes:

[0061] An acquisition unit, configured to acquire the environmental information of the current driving scene and match the corresponding dual-point light source image recognition and optimization scheme based on the environmental information.

[0062] Specifically, the matching of the corresponding dual-point light source image recognition and optimization scheme based on the environmental information includes: generating a binary array based on the light intensity and temperature information indicated by the environmental information; performing a matching search for the dual-point light source image recognition and optimization scheme in a pre-constructed scheme library based on the binary data to obtain the dual-point light source image recognition and optimization scheme with the closest similarity.

[0063] In the embodiment of the present invention, the light and temperature of the environment will affect the performance and imaging effect of the optical system. Based on this feature, when performing dual-point light source image recognition and optimization, the two influencing factors of the scheme need to be fully considered to facilitate the execution of the best imaging optimization scheme for the current scene. Based on this, the scheme of the present invention first collects the light intensity and temperature information of the scene, and then generates the corresponding binary array based on the collected information to facilitate the subsequent optimal scheme matching based on the binary array.

[0064] Further, the method further includes: constructing the solution library, including: performing simulation adaptive adjustment on light intensity and temperature information based on a laboratory scenario, and constructing a binary array corresponding to the light intensity and temperature information for each adjustment to construct a first array sample set; collecting the light intensity and temperature information of the corresponding driving scenario by each data communication vehicle, and constructing a binary array corresponding to the light intensity and temperature information for each data collection to construct a second array sample set; integrating the first array sample set and the second array sample set to obtain an integrated array sample set; constructing a training sample set based on the binary arrays corresponding to the double-point light source images in the array sample set; performing light point recognition processing on each double-point light source image based on the training sample, and determining the spot halo size of each double-point light source image based on the recognition result; determining an optimized camera parameter solution for the corresponding scenario based on the spot halo size of each double-point light source image, and constructing a solution library based on all the results.

[0065] In an embodiment of the present invention, for the integrity of the solution library, the solution of the present invention collects sample data from two aspects, namely, a laboratory scenario and a real scenario. In the laboratory scenario, a simulated double-point light source is constructed, and then the light intensity and temperature information of the scenario are adaptively adjusted, and a target double-point light source image is collected once after each adjustment. In the actual scenario, real-scenario images are collected through the in-vehicle terminals of each connected server, and then vehicle headlight image annotation is performed on the collected real-scenario images. The annotated images are used as sample images, and the light intensity information and temperature information at the image sampling moment are determined synchronously. A complete solution library is constructed through the training samples obtained from the laboratory scenario and the training samples based on the real scenario. Further, the solution library can also be continuously updated based on new scenario images subsequently.

[0066] Further, the performing light point recognition processing on each double-point light source image based on the training sample and determining the light source halo size of each double-point light source image based on the recognition result includes: performing grayscale processing on the double-point light source image to obtain a grayscale image; performing smoothing filtering processing on the grayscale image, and performing threshold segmentation processing on the smoothed grayscale image to obtain a segmented image; performing spot area marking on the segmented image to output a spot-marked image; identifying a halo area in the spot-marked image, and calculating the halo area.

[0067] In an embodiment of the present invention, it specifically includes the following steps:

[0068] 1) Perform grayscale processing on the image: Grayscale processing of an image is the process of converting a color image into a grayscale image, that is, converting the RGB (red, green, blue) channel values of each pixel into a single grayscale value. This processing can simplify the image processing process, reduce the computational amount, and retain the main features of the image. First, obtain the color image data from an input source (such as a camera, a file, etc.). A color image usually consists of three channels: red, green, and blue. For each pixel, the values of its RGB channels are weighted and averaged according to certain weights to obtain a grayscale value. The corresponding grayscale formula is: Gray = 0.299 * R + 0.587 * G + 0.114 * B. These weights are determined according to the sensitivity of the human eye to different colors. The grayscale value usually ranges from 0 to 255, representing different grayscale levels from black to white. After grayscale processing, it may be necessary to adjust the range of the grayscale values to ensure that the grayscale values of all pixels are within an appropriate range. Recombine the grayscale values of each pixel into a grayscale image. A grayscale image has only one channel, and each pixel has only one grayscale value. Therefore, compared with a color image, a grayscale image occupies less storage space. Finally, the grayscale image can be displayed on the screen or saved as a file for subsequent processing or analysis. Through grayscale processing, the complexity of image processing can be simplified, the computational amount can be reduced, and the main features of the image can be retained.

[0069] 2) Image smoothing filtering processing: Image smoothing filtering is a commonly used image processing technique for reducing noise in an image and smoothing the details of the image. Smoothing filtering can help improve the image quality and make the image easier to process and analyze. First, use the image data after grayscale processing as the image data to be smoothed filtered. Select an appropriate smoothing filter. Commonly used smoothing filters include the mean filter, the Gaussian filter, the median filter, etc. Different filters have different smoothing effects and applicable scenarios. Apply the selected filter to each pixel of the image. The filter is usually a small matrix, and the smoothing effect is achieved by sliding the matrix on the image and performing weighted averaging on the surrounding pixels. When sliding the filter, the processing of boundary pixels needs to be considered. Use different boundary processing methods, such as filling boundary pixels, ignoring boundary pixels, or using specific boundary processing algorithms. Adjust the parameters of the filter as needed, such as the size and weights of the filter, to achieve the best smoothing effect. Recombine the filtered pixels into a smoothed image. The smoothed image usually has less noise and details and is more suitable for subsequent processing or analysis. Finally, save the smoothed image as a file for subsequent processing or analysis. Through image smoothing filtering processing, the noise in the image can be effectively reduced, the details of the image can be smoothed, and the image quality can be improved.

[0070] 3) Image threshold segmentation processing: First, use the image after smoothing filtering as the image data to be threshold-segmented. According to the characteristics and requirements of the image, select an appropriate threshold. The choice of threshold has a great impact on the segmentation result, and the optimal threshold can be determined by methods such as histogram analysis and experimental method. Apply the selected threshold to each pixel of the image, and divide the pixels into two categories: above the threshold and below the threshold. This step can obtain a binary image that only contains two pixel values. Perform post-processing on the segmented image, such as removing small regions, filling holes, connecting adjacent regions, etc., to obtain a more accurate segmentation result and obtain the spot segmentation result. Finally, the segmented image can be displayed on the screen or saved as a file for subsequent processing or analysis.

[0071] 4) Perform spot area marking: In the image after threshold segmentation, the spot area is the segmented area with specific characteristics. By analyzing the segmented image, find the position and shape of the spot area. Draw a bounding box or contour around the spot area to highlight the position and shape of the spot area, which can be achieved by drawing rectangles, polygons or other shapes on the image. To more clearly label the spot area, add labels or annotations near the spot area to describe the content of the target.

[0072] Furthermore, the spot area marking of the segmented image and the output of the spot marking image include: performing erosion and / or dilation processing on the segmented image to obtain the spot marking area; in the spot marking area, perform connected component analysis, and based on the analysis result, perform spot marking on the positions of the two light sources in the double-point light source image, and output the spot marking image; the identification of the halo area in the spot marking image and the calculation of the halo area based on the area include: performing secondary threshold segmentation on the spot marking image to obtain the halo area; determining the single-pixel area scale based on the light source size of the double-point light source; counting the number of pixel points in the halo area, and calculating the area of the halo area based on the number of pixel points and the single-pixel area scale.

[0073] In the embodiment of the present invention, in the obtained marked image, the halo of the spot area is often very large, resulting in a very small distance between the two optoelectronics, and reducing the discrimination effect of the double-point light source. To solve this situation, the solution of the present invention needs to identify the size of the halo, and perform spot center area and halo area segmentation by performing threshold segmentation processing again. Since the area of the spot center area is known (related to the light source size), the single-pixel area scale is determined by the known area of the spot center, and then the size of the halo can be determined based on the number of pixel points contained in the segmented halo area.

[0074] The execution unit is used to adjust the camera parameters of the electronic rearview mirror based on the matched dual-point light source image recognition optimization solution, and perform image acquisition based on the camera with adjusted parameters to obtain an image to be recognized.

[0075] Specifically, halo refers to the blurry or bright effect that appears around bright areas in images or photos. Long exposure time means that the lens receives more light, especially in the case of backlight or strong light source. Long exposure time will increase the possibility of halo effect. Long exposure time will cause light to reflect and scatter multiple times inside the lens, aggravate the halo effect, and make the halo more obvious. Short exposure time means that the lens receives less light, which reduces the possibility of halo effect. Short exposure time can reduce the number of reflections of light inside the lens, reduce the halo effect, and make the halo less obvious. However, too short exposure time will cause insufficient light to be received by the camera's photosensitive element, making the image too dark and the details cannot be clearly displayed. Therefore, if you want to reduce the influence of halo, you must ensure that the collected image has sufficient ability to identify the characteristics of the spot area, and you need to adjust the appropriate exposure time. Based on this, the point light source image recognition optimization scheme of the present invention includes: a camera parameter optimization scheme and an image optimization scheme; the camera parameter optimization scheme is an exposure time optimization scheme.

[0076] The optimization unit is used to perform optimization processing on the image to be identified based on the matched dual-point light source image recognition optimization scheme to obtain an optimized image.

[0077] Specifically, in addition to improving halo by shortening the exposure time during image acquisition, the solution of the present invention will also improve halo by adjusting the gain during the image backend processing. The halo effect is mitigated by adjusting parameters such as contrast and brightness. Gamma value optimization refers to adjusting the Gamma value in image processing to optimize the brightness and contrast of the image. In digital image processing, the Gamma value is usually used to correct the nonlinear response of the display device so that the image can more accurately show the brightness and color of the original scene when displayed. Gamma value optimization can help adjust the overall brightness and contrast of the image, making the image look clearer and more natural.

[0078] Furthermore, the "ltm optimization" in camera gain refers to "Local Tone Mapping optimization". Local Tone Mapping is a technique used to enhance local details and contrast in images, commonly used in image processing and computational photography. The Local Tone Mapping technique aims to adjust the local contrast of an image, making the details in the image clearer and more prominent. By applying different tone mapping functions in different regions, the enhancement of local details in the image can be achieved, thereby improving the visual effect of the image. Applying Local Tone Mapping optimization in camera gain can help improve the quality and detail performance of the image, especially in images taken under low light conditions. This optimization can make the details in the dark and bright parts of the image better presented, thus enhancing the overall visual effect.

[0079] Furthermore, Sharpness optimization refers to enhancing the sharpness and details of an image through a series of techniques and methods, making the image look clearer and sharper. The sharpness of an image refers to the clarity of the edges and details of objects in the image, which affects the visual quality and recognition ability of the image. The solution of the present invention executes the Sharpness optimization method through the following steps:

[0080] 1) Sharpening Filters: By applying sharpening filters (such as Laplace filters) to enhance the edges and details of the image, highlighting the subtle changes in the image, thereby improving the sharpness of the image.

[0081] 2) Local Contrast Enhancement: By enhancing the contrast of local regions in the image, the details in the image can be made more prominent, thereby improving the sharpness and visual effect of the image.

[0082] 3) Super-Resolution Reconstruction: By using super-resolution technology to reconstruct a low-resolution image into a high-resolution image, thereby enhancing the sharpness and details of the image.

[0083] 4) Deblurring: Removing the blurring effect in the image, restoring the sharpness and details of the image, making the image look clearer and sharper.

[0084] 5) Local Detail Enhancement: Enhancing the details of specific regions or features in the image, highlighting the important details in the image, and improving the sharpness and quality of the image.

[0085] In the embodiments of the present invention, by applying the above Sharpness optimization method, the visual quality of the image can be improved, making the image clearer and sharper, thereby significantly improving the halo phenomenon.

[0086] A pushing unit, configured to push the optimized image to the in-vehicle display module to perform real-time display of the optimized image.

[0087] Specifically, based on the environmental information of the current scene, after selecting the corresponding optimization scheme, the video stream information collected is optimized in real time, and then the optimization result is displayed in real time, so that the display module on the in-vehicle terminal always displays the most optimized dual-point light source imaging image, ensuring the user experience and also ensuring driving safety.

[0088] The embodiments of the present invention also provide a computer-readable storage medium, on which instructions are stored, and when running on a computer, the computer is caused to execute the above-mentioned electronic rearview mirror dual-point light source image recognition optimization method.

[0089] Those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions to cause a single-chip microcomputer, a chip, or a processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other various media that can store program codes.

[0090] The optional embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. In addition, it should be noted that, among the various specific technical features described in the above specific embodiments, they can be combined in any appropriate manner without conflict. To avoid unnecessary repetition, the embodiments of the present invention will not describe various possible combination methods separately.

[0091] In addition, any combination can be made between various different embodiments of the present invention, as long as it does not violate the idea of the embodiments of the present invention, and it should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for optimizing image recognition of dual-point light sources in an electronic rearview mirror, characterized in that: The method comprises: Collecting environmental information of the current driving scene, and generating a binary array based on light intensity and temperature information indicated by the environmental information; Based on the binary array, a matching search is performed on a dual-point light source image recognition optimization solution in a pre-built solution library to obtain a dual-point light source image recognition optimization solution with the closest similarity; The construction rules of the solution library include: Performing simulated adaptive adjustment of light intensity and temperature information based on a laboratory scene, and constructing a binary array corresponding to the light intensity and temperature information based on each adjustment to construct a first array sample set; Based on the light intensity and temperature information of the corresponding driving scene collected by each data communication vehicle, a binary array of the corresponding light intensity and temperature information is constructed based on each data collection, and a second array sample set is constructed; Integrate the first array sample set and the second array sample set to obtain an integrated array sample set; Constructing a training sample set based on the dual point light source images corresponding to each binary array in the integrated array sample set; Based on the training samples, light spot recognition processing is performed on each dual-point light source image, and the spot halo size of each dual-point light source image is determined based on the recognition result; wherein, The step of performing light spot recognition processing on each dual-point light source image based on the training samples and determining the light source halo size of each dual-point light source image based on the recognition result includes: Performing grayscale processing on the dual point light source image to obtain a grayscale image; Performing smoothing filtering on the grayscale image, and performing threshold segmentation on the grayscale image subjected to the smoothing filtering to obtain a segmented image; Mark the spot areas of the segmented image and output the spot marked image; Identifying the halo region in the spot mark image and calculating the area of ​​the halo region; The step of marking the spot regions of the segmented image and outputting the spot marked image comprises: Performing erosion and / or dilation processing on the segmented image to obtain a spot marking area; In the spot marking area, performing connected component analysis, performing spot marking of two light source positions in the dual-point light source image based on the analysis result, and outputting a spot marking image; The step of identifying the halo region in the spot mark image and calculating the halo region area comprises: Perform secondary threshold segmentation on the spot-marked image to obtain the halo region; Determine the single pixel area scale based on the light source size of the dual point light sources; Counting the number of pixels in the halo area, and calculating the area of ​​the halo area based on the number of pixels and the single pixel area scale; Determine the camera parameter optimization solution for the corresponding scene based on the spot halo size of each dual-point light source image, and build a solution library based on all the results; Based on the matched dual-point light source image recognition optimization solution, the camera parameters of the electronic rearview mirror are adjusted, and image acquisition is performed based on the camera with adjusted parameters to obtain the image to be recognized; Based on the matched dual-point light source image recognition optimization scheme, performing optimization processing on the image to be recognized to obtain an optimized image; Push the optimized image to the vehicle display module and perform real-time display of the optimized image.

2. The method according to claim 1, characterized in that The dual-point light source image recognition optimization solution includes: Camera parameter optimization solution and image optimization solution; among them, The camera parameter optimization scheme and the image optimization scheme are respectively an exposure time optimization scheme and a camera gain optimization scheme; The camera gain optimization scheme includes: Gamma value optimization, ltm optimization, sharpness optimization.

3. The method according to claim 2, characterized in that The dual-point light source image recognition optimization scheme based on matching is used to adjust the camera parameters of the electronic rearview mirror, and to perform image acquisition based on the camera after the parameter adjustment to obtain the image to be recognized, including: Based on the exposure time optimization scheme, adjusting the exposure parameters of the camera of the electronic rearview mirror; Based on the camera with adjusted exposure parameters, scene images are continuously collected during vehicle driving. When a light source target appears in the collected image, video frame capture processing is performed, and the frame image containing the light source target is used as the image to be recognized.

4. The method according to claim 2, characterized in that: The matching-based dual-point light source image recognition optimization scheme performs optimization processing on the image to be recognized to obtain an optimized image, including: Gamma value optimization, LTM optimization and Sharpness optimization are performed on the image to be recognized in sequence, and the optimized image is displayed in real time based on the vehicle display module.

5. An electronic rearview mirror dual point light source image recognition optimization system, characterized in that: The system comprises: A collection unit, used to collect environmental information of the current driving scene, and generate a binary array based on light intensity and temperature information indicated by the environmental information; Based on the binary array, a matching search is performed on a dual-point light source image recognition optimization solution in a pre-built solution library to obtain a dual-point light source image recognition optimization solution with the closest similarity; The construction rules of the solution library include: Performing simulated adaptive adjustment of light intensity and temperature information based on a laboratory scene, and constructing a binary array corresponding to the light intensity and temperature information based on each adjustment to construct a first array sample set; Based on the light intensity and temperature information of the corresponding driving scene collected by each data communication vehicle, a binary array of the corresponding light intensity and temperature information is constructed based on each data collection, and a second array sample set is constructed; Integrate the first array sample set and the second array sample set to obtain an integrated array sample set; Constructing a training sample set based on the dual point light source images corresponding to each binary array in the integrated array sample set; Based on the training samples, light spot recognition processing is performed on each dual-point light source image, and the spot halo size of each dual-point light source image is determined based on the recognition result; wherein, The step of performing light spot recognition processing on each dual-point light source image based on the training samples and determining the light source halo size of each dual-point light source image based on the recognition result includes: Performing grayscale processing on the dual point light source image to obtain a grayscale image; Performing smoothing filtering on the grayscale image, and performing threshold segmentation on the grayscale image subjected to the smoothing filtering to obtain a segmented image; Mark the spot areas of the segmented image and output the spot marked image; Identifying the halo region in the spot mark image and calculating the area of ​​the halo region; The step of marking the spot regions of the segmented image and outputting the spot marked image comprises: Performing erosion and / or dilation processing on the segmented image to obtain a spot marking area; In the spot marking area, performing connected component analysis, performing spot marking of two light source positions in the dual-point light source image based on the analysis result, and outputting a spot marking image; The step of identifying the halo region in the spot mark image and calculating the halo region area comprises: Perform secondary threshold segmentation on the spot-marked image to obtain the halo region; Determine the single pixel area scale based on the light source size of the dual point light sources; Counting the number of pixels in the halo area, and calculating the area of ​​the halo area based on the number of pixels and the single pixel area scale; Determine the camera parameter optimization solution for the corresponding scene based on the spot halo size of each dual-point light source image, and build a solution library based on all the results; An execution unit, used to adjust the camera parameters of the electronic rearview mirror based on the matched dual-point light source image recognition optimization scheme, and perform image acquisition based on the camera with adjusted parameters to obtain an image to be recognized; An optimization unit, configured to perform optimization processing on the image to be recognized based on a matched dual-point light source image recognition optimization scheme to obtain an optimized image; The push unit is used to push the optimized image to the vehicle display module and perform real-time display of the optimized image.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the electronic rearview mirror dual-point light source image recognition optimization method described in any one of claims 1 to 4.

Images