Head-up display windshield quality detection method and system based on machine vision, computer readable storage medium and computer program product

Abstract

The invention belongs to the field of quality detection of mobile automobile windshields, and discloses a quality detection method of a head-up display windshield based on machine vision, which comprises the following steps: acquiring an original image of an original projection virtual image formed by a standard image projected by a head-up display system through the windshield, and carrying out graying processing on the original image to obtain a gray image; a corresponding grey-scale map is obtained; extracting an ROI (Region of Interest) area, namely a virtual image area, from the grey-scale map, performing dual-threshold processing on the image from which the virtual image area is extracted so as to segment a background area and the virtual image area, highlighting edge features of a virtual image in the background area and the virtual image area, and obtaining a thresholding image; and measuring each physical parameter of the thresholding image, comparing the difference between the measurement result and the corresponding parameter of the standard image, and if the difference is within a preset threshold range, determining that the imaging quality of the head-up display windshield is qualified, otherwise, determining that the imaging quality of the head-up display windshield is unqualified. The stability and efficiency of HUD imaging quality measurement of the windshield are improved, and meanwhile the cost of the whole detection system is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the field of quality inspection of mobile vehicle windshields, and more specifically, relates to a quality inspection method, system, computer-readable storage medium, and computer program product for a head-up display windshield based on machine vision. Background Art

[0002] With the development of the automotive industry, the quality control of windshields has become increasingly important, especially in improving driving safety, driving comfort, and the performance of in-vehicle technical systems. Currently, many researchers have designed various windshield quality inspection schemes using digital measurement technologies such as contact measurement, laser 3D scanning, and other non-contact sensing technologies to improve the measurement accuracy of thin-walled parts. For example, Xia Qilong et al. (Patent CN112432627A) designed a contact detection mechanism with the same curved surface shape as the upper and lower surfaces of a standard glass part, and by processing the detection data of the contact mechanism, obtained the radian deviation and thickness deviation, thereby realizing the quality inspection of the windshield. Wang Cheng et al. (Patent CN114295044A) developed a method for detecting the quality of automotive windshields using laser 3D technology, and by comparing the three-dimensional reconstruction with the product model, judged the quality difference of the windshield.

[0003] Although the existing measurement technologies can achieve accurate detection of the appearance and geometric dimensions of windshields to a certain extent, they are usually limited to the inspection of surface quality and shape defects, and it is difficult to comprehensively evaluate the performance of windshields in actual applications, especially the impact brought by advanced driver assistance systems (ADAS) and head-up display systems (HUD). Specifically, minor defects in the windshield, such as bending, optical distortion, uneven thickness, etc., may have a significant impact on the image quality, display accuracy, and driver's field of view of the head-up display system, thereby affecting the driving experience and safety.

[0004] However, the existing technologies have not been able to provide an effective detection method that can comprehensively evaluate the impact of windshield quality defects on the functions of the head-up display system. Summary of the Invention

[0005] In view of the above defects or improvement requirements of the existing technology, the present invention provides a quality inspection method, system, computer-readable storage medium, and computer program product for a head-up display windshield based on machine vision, aiming to solve the problem of detecting the head-up display quality of the windshield.

[0006] To achieve the above object, according to one aspect of the present invention, a quality inspection method for a head-up display windshield based on machine vision is provided, including:

[0007] Obtain the original image of the original projected virtual image formed by the head-up display system projecting the standard image through the windshield, and perform grayscale processing on the original image to obtain the corresponding grayscale image;

[0008] Extract the ROI region, i.e., the virtual image region, from the grayscale image, and perform double-threshold processing on the image from which the virtual image region has been extracted to segment the background region and the virtual image region, and highlight the edge features of the virtual image therein to obtain a thresholded image;

[0009] Measure various physical parameters of the thresholded image, and compare the measurement results with the corresponding parameters of the standard image. If the difference is within the preset threshold range, the imaging quality of the head-up display windshield is qualified; otherwise, it is unqualified.

[0010] Furthermore, it also includes using a similarity measurement algorithm to measure the difference between the standard image and the original image to determine the specific defect type. The similarity expression formula is as follows:

[0011] S p (M,T)=|f(M p )-f(T p )|

[0012]

[0013] Among them, S T (M,T) is the global similarity between the standard image M and the original image T. The smaller its value, the higher the overall imaging quality of the windshield. S p (M,T) is the local similarity between the standard image M and the original image T for the p-th block. The smaller its value, the higher the local imaging quality of the windshield. The number of blocks of the standard image M and the original image T is both P, p = 1 to P, f(M p ) is the feature vector of the p-th block of the standard image M, and f(T p ) is the feature vector of the p-th block of the original image T;

[0014] The method for judging the specific quality category is as follows:

[0015]

[0016] Among them, a is an empirical value, S p (M,T)[A] represents all S p (M,T), and S p (M,T)[I] represents any one S p (M,T).

[0017] Further, the feature vector of the original image T is directly composed of the physical parameters of the thresholded image obtained by measurement; the feature vector of the standard image M is composed of the parameters corresponding to the physical parameters of the original image T.

[0018] Further, before extracting the ROI region from the grayscale image, a sliding window process is also performed on the grayscale image:

[0019] For each pixel (x, y) in the grayscale image, the median of its neighborhood median(I(x ′ , y ′ )) is selected as the new value I filtered (x, y).

[0020] Further, for the head-up display quality detection of the windshield, it is characterized in that it includes a projection background board, an industrial camera, a tooling bracket, a head-up display device, a memory, a processor, and a computer program stored on the memory;

[0021] The tooling bracket is used to fix the windshield, the industrial camera, and the head-up display device to simulate the layout position and angle of the car cockpit;

[0022] The head-up display device is used to output a test pattern to the windshield, so as to form a projection virtual image on the projection background board;

[0023] The industrial camera is used to collect the projection virtual image;

[0024] The processor executes the computer program to process the projection virtual image according to the quality detection method described in any one of the preceding items, so as to detect the quality of the head-up display windshield.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the quality detection method described in any one of the preceding items.

[0026] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, it implements the quality detection method described in any one of the preceding items.

[0027] Generally speaking, compared with the prior art, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0028] 1. Through the virtual image detection and measurement method based on machine vision, the detection system of the present invention can judge the quality problem of the windshield through the collected image data, solve the problem that the traditional measurement method depends on the cooperation of structured light scanning and laser thickness measurement technology and cannot efficiently evaluate the quality of the windshield. This comprehensive solution improves the stability and efficiency of measurement, and at the same time greatly reduces the cost of the entire detection system.

[0029] 2. In order to further and specifically determine the specific type of the quality problem of the windshield head-up display, the present invention further proposes a similarity improvement algorithm for the characteristics of the head-up display image. This algorithm can calculate the similarity between the actual test image (i.e., the original image) collected by the industrial camera and the theoretical template image (i.e., the standard image), and can accurately judge the type of the output image quality. It can not only study and analyze the overall characteristics of the image, but also realize the comparison and calculation of the local characteristics of the image, and better evaluate the image quality of the windshield head-up display.

[0030] 3. This similarity improvement algorithm can be implemented either by independently collecting feature vectors or by directly using the feature vectors composed of the measured physical parameters of the virtual image, and can be widely applied to the quality detection of the windshield head-up display under different experimental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is the overall system measurement flow chart.

[0032] Figure 2 is a schematic diagram of the windshield quality detection system.

[0033] Figure 3 is the similarity measurement algorithm flow chart. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] The present invention proposes a measurement method and system based on machine vision, aiming to realize the quality detection of the windshield in the head-up display system. This system combines a head-up display device and an industrial camera, and conducts quality detection on the target workpiece by simulating the actual environment of the car cockpit. Specifically, as Figure 2 shown, the head-up display device 5 is used to output a test pattern, and this test pattern is deformed through two reflections by the internal free-form mirror and the external curved windshield 2; the industrial camera 3 realizes high-precision data acquisition of the projected virtual image, and judges the quality of the windshield 2 by analyzing the deformation of the projected virtual image. Before measurement, the system completes the input of the test pattern and the calibration of the industrial camera 3, then collects the image data of the projected virtual image, and uses the data processing unit to apply an efficient algorithm for processing, and finally outputs a high-precision quality evaluation model. This method significantly improves the reliability and accuracy of product quality evaluation.

[0036] Before the experiment starts, the windshield 2, the industrial camera 3, and the head-up display device 5 are integrated onto the tooling bracket 4 according to the layout position and angle of the simulated automotive cockpit. The projection background board 1 is placed directly in front of the windshield 2 at the virtual image distance. Before the system is started, the calibration and position correction of the industrial camera 3 are completed.

[0037] The present invention preferably obtains a method for detecting the quality of a head-up display windshield based on machine vision. As shown in the flowchart of the appendix Figure 1 shown, it includes the following steps:

[0038] The industrial camera 3 collects the original virtual image projected by the head-up display device 5, grayscales the image, and obtains the corresponding grayscale image.

[0039] The grayscale image is subjected to median filtering to eliminate noise, light sources, and abnormal point interference in the image and improve the image quality.

[0040] The ROI region (virtual image region) in the virtual image is extracted. The virtual image region contains defect information to be detected, and the background region and the virtual image region are segmented by a threshold method.

[0041] The image is subjected to double-threshold processing to obtain a defect image with clear features, highlighting the virtual image edge features and providing a basis for subsequent analysis.

[0042] For the preprocessed image, physical parameters related to the virtual image (such as parameters that can be used to judge the imaging quality, such as size and deformation degree) are accurately measured, thereby obtaining the virtual image quality parameters.

[0043] For the thresholded image containing defects and the quality parameter information, a similarity measurement algorithm is used for measurement, or a new quality parameter is directly detected and then the similarity measurement algorithm is used for measurement, and a final quality evaluation model is generated.

[0044] Optionally, the projection background board 1 is installed directly in front of the windshield 2 to receive the virtual image projection and serve as a reference board for auxiliary measurement. The windshield 2 is the target workpiece to be detected and is also part of the optical path of the head-up display system. The industrial camera 3 is used to collect the virtual image data of the head-up display system and is installed on the tooling bracket 4. The tooling bracket 4 is assembled from aluminum profiles and is used to fix the industrial camera 3, the windshield 2, and the head-up display device 5 to simulate the layout position and angle of the automotive cockpit.

[0045] Optionally, the head-up display device 5 serves as an image source and is responsible for projecting a selected test pattern. The data processing unit is connected to the industrial camera 3 and the head-up display device 5 and is used to integrate the virtual image data and perform processing and fusion.

[0046] Optionally, during the camera calibration process, the Zhang Zhengyou camera calibration method is adopted to calibrate by calculating the internal and external parameters of the camera. The internal parameter matrix of the camera can be expressed as:

[0047]

[0048] where f x and f y represent the focal lengths respectively, and c x and c y represent the image center coordinates.

[0049] Optionally, in step S20, during the grayscale conversion process, use I gray = 0.299×I R + 0.587×I G + 0.114×I B to convert the original RGB image into a grayscale image, where I R , I G , I B represent the pixel values of the red, green, and blue channels respectively.

[0050] Optionally, the specific method for median filtering the grayscale image is to process the image using a sliding window. For each pixel (x, y), select the median median(I(x′, y′)) within its neighborhood as the new value I filtered (x, y) of this pixel, that is, I filtered (x, y) = median(I(x ′ , y ′ ))

[0051] Optionally, ROI extraction uses a threshold-based image segmentation method. For example, the Otsu method is used to automatically select the optimal threshold.

[0052] Optionally, the method for double-threshold processing of the image is to use the Canny edge detection algorithm to extract the edges of the image. The calculation steps of the Canny algorithm include: Gaussian filtering; calculating the gradient intensity non-maximum suppression to eliminate non-edge pixels; double-threshold processing to determine the weak and strong edges of the edges. G x is the gradient in the x direction, and G y is the gradient in the y direction.

[0053] Optionally, during the process of measuring the parameters related to the virtual image, first perform surface fitting on the virtual image, use the least squares method to fit the three-dimensional surface of the virtual image, and then measure the relevant physical parameters based on the fitted three-dimensional surface of the virtual image. The physical parameters measured here can be histograms, Hu moments, SIFT features, VLAD, BRISK, SURF, etc. For details, please refer to the following table (since the relevant features are conventional features, not exhaustively listed):

[0054]

[0055] Optionally, a similarity metric algorithm is used to measure the difference between the standard image and the original image, and the formula can be expressed as:

[0056] S p (M, T) = |f(M p ) - f(T p )|

[0057]

[0058] where S T (M, T) is the global similarity between the standard image M and the original image T. The smaller its value, the higher the HUD imaging quality. S p (M, T) is the local similarity of the p-th block between the standard image M and the original image T. The number of blocks of both the standard image M and the original image T is P, p = 1 to P. f(M p ) is the feature vector of the p-th block of the standard image M, and f(T p ) is the feature vector of the p-th block of the original image T. The feature vector can be calculated or detected separately, or directly composed of the physical parameters measured previously.

[0059] Serial number Global similarity Local similarity Result type 1 <![CDATA[S T (M,T) = 0]]> <![CDATA[S p (M,T)[A] = 0]]> Qualified 2 <![CDATA[S T (M,T) = 0]]> <![CDATA[S p (M,T)[A]≠0]]> Offset 3 <![CDATA[S T (M,T) = 0]]> <![CDATA[S p (M,T)[A]≠0]]> Rotation 4 <![CDATA[S T (M,T)≥a]]> <![CDATA[S p (M,T)[I] = 0]]> Distortion 5 <![CDATA[0<S T (M,T)<a]]> <![CDATA[S p (M,T)[A]≠0]]> Ghosting

[0060] a is an empirical value, preferably 0.3.

[0061] Generate the model for final quality assessment:

[0062]

[0063] where a is an empirical value, and S p (M, T)[A] represents all S p (M, T), and S p (M, T)[I] represents any one of S p (M, T).

[0064] A windshield quality detection system based on machine vision proposed by the present invention can detect the optical performance and geometric shape of the windshield in real time by projecting virtual image data and combining advanced computer vision algorithms, and can accurately judge its impact on the head-up display system. Compared with the prior art, the present invention can not only detect the external defects of the windshield, but also effectively evaluate its potential impact on the automotive driving assistance system, thereby providing a more comprehensive and accurate quality control solution.

[0065] This system combines a head-up display device and an industrial camera to detect the quality of the target workpiece by simulating the actual environment of the automotive cockpit. Specifically, the head-up display device is used to output a test pattern, and the test pattern is deformed through two reflections by the internal free-form mirror and the external curved windshield; while the industrial camera realizes high-precision data acquisition of the projected virtual image, and judges the quality of the windshield by analyzing the deformation of the projected virtual image. Before measurement, the system completes the input of the test pattern and the calibration of the camera, then collects the image data of the projected virtual image, and uses the data processing unit to apply an efficient algorithm for processing, and finally outputs a high-precision quality evaluation model, significantly improving the reliability and accuracy of product quality evaluation.

[0066] It is easy for those skilled in the art to understand that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A quality inspection method for a head-up display windshield based on machine vision, characterized in that: include: Obtaining an original image of an original projected virtual image formed by a standard image projected by a head-up display system through a windshield, and gray-processing the original image to obtain a corresponding grayscale image; Extract the ROI region, i.e., the virtual image region, from the grayscale image, perform double threshold processing on the image from which the virtual image region is extracted, so as to segment the background region and the virtual image region, and highlight the edge features of the virtual image therein, thereby obtaining a thresholded image; The various physical parameters of the thresholded image are measured, and the measurement results are compared with the corresponding parameters of the standard image. If the difference is within the preset threshold range, the imaging quality of the head-up display windshield is qualified, otherwise it is unqualified.

2. The quality inspection method of a head-up display windshield based on machine vision according to claim 1, characterized in that: It also includes using a similarity measurement algorithm to measure the difference between the standard image and the original image to determine the specific defect type. The similarity expression formula is as follows: S p (M,T)=|f(M p )-f(T p )| Among them, S T (M, T) is the global similarity between the standard image M and the original image T. The smaller the value, the higher the overall imaging quality of the windshield. p (M,T) is the local similarity of the pth block between the standard image M and the original image T. The smaller the value, the higher the local imaging quality of the windshield. The number of blocks of the standard image M and the original image T is P, p = 1 ~ P, f(M p ) is the feature vector of the pth block of the standard image M, f(T p ) is the feature vector of the pth block of the original image T; The method for determining specific quality categories is as follows: Among them, a is the empirical value, S p (M,T)[A] represents all S p (M,T),S p (M,T)[I] represents any S p (M,T).

3. The quality inspection method of a head-up display windshield based on machine vision according to claim 2 is characterized in that: The feature vector of the original image T is directly composed of the various physical parameters of the thresholded image obtained by measurement; the feature vector of the standard image M is composed of the parameters corresponding to the various physical parameters of the original image T.

4. The quality inspection method of a head-up display windshield based on machine vision according to claim 1, characterized in that: Before extracting the ROI area from the grayscale image, the grayscale image is also subjected to sliding window processing: For each pixel (x, y) in the grayscale image, select the median (I(x ′ ,y ′ )) as the new value of the pixel I filtered (x,y).

5. A head-up display windshield quality inspection system based on machine vision, used for head-up display quality inspection of windshield (2), characterized in that: It comprises a projection background plate (1), an industrial camera (3), a tooling bracket (4), a head-up display device (5), a memory, a processor and a computer program stored in the memory; The tooling bracket (4) is used to fix the windshield (2), the industrial camera (3) and the head-up display device (5) to simulate the layout position and angle of the car cabin; The head-up display device (5) is used to output a test pattern to the windshield (2), thereby forming a projected virtual image on the projection background plate (1); The industrial camera (3) is used to collect the projected virtual image; The processor executes the computer program to process the projected virtual image according to the quality detection method according to any one of claims 1 to 3 so as to detect the quality of the head-up display windshield.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the quality detection method according to any one of claims 1 to 3 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the quality detection method according to any one of claims 1 to 3 is implemented.

Images