A method, system, computer readable storage medium and computer program product for quality detection of a head-up display windshield based on machine vision

Abstract

The application belongs to the field of quality detection of mobile automobile windshield, and discloses a quality detection method of head-up display windshield based on machine vision, which comprises the following steps: obtaining a standard image projected by a head-up display system and an original image of a primary projection virtual image formed by passing through a windshield, and carrying out gray processing on the original image to obtain a corresponding gray image; extracting an ROI region, i.e., a virtual image region, from the gray image, carrying out double-threshold processing on the image from which the virtual image region is extracted, to segment a background region and the virtual image region, highlight the edge features of the virtual image, and obtain a threshold image; measuring various physical parameters of the threshold image, and comparing the measurement results with corresponding parameters of the standard image; if the difference is within a preset threshold range, the imaging quality of the head-up display windshield is qualified, otherwise, the imaging quality of the head-up display windshield is unqualified. The application improves the stability and efficiency of the HUD imaging quality measurement of the windshield, and greatly reduces the cost of the whole detection system.

Description

Technical Field

[0001] This invention belongs to the field of quality inspection of mobile car windshields, and more specifically, relates to a method, system, computer-readable storage medium, and computer program product for quality inspection of head-up display windshields based on machine vision. Background Technology

[0002] With the development of the automotive industry, the quality control of windshields has become increasingly important, especially in improving driving safety, driving comfort, and the performance of in-vehicle technology systems. Currently, many researchers are employing digital measurement technologies, such as contact measurement, laser 3D scanning, and other non-contact sensing technologies, to design various windshield quality inspection schemes to improve the measurement accuracy of thin-walled components. For example, Xia Qilong et al. (patent CN112432627A) designed a contact detection mechanism with the same curved surface shape as the upper and lower surfaces of a standard glass component. By processing the detection data from the contact mechanism, they obtained the curvature deviation and thickness deviation, thereby achieving quality inspection of the windshield. Wang Cheng et al. (patent CN114295044A) developed a method for inspecting the quality of automotive windshields using laser 3D technology. By comparing the 3D reconstruction with the product model, they determined the quality differences of the windshield.

[0003] While existing measurement technologies can achieve a certain degree of precision in inspecting the appearance and geometry of windshields, they are typically limited to checking surface quality and shape defects. They struggle to comprehensively assess the windshield's performance in real-world applications, particularly its impact on Advanced Driver Assistance Systems (ADAS) and Head-Up Displays (HUDs). Specifically, even minor defects in the windshield, such as warping, optical distortion, or uneven thickness, can significantly affect the image quality, display accuracy, and driver visibility of the HUD, thereby impacting the driving experience and safety.

[0004] However, existing technologies have not yet provided an effective detection method that can comprehensively assess the impact of windshield quality defects on the function of head-up display systems. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the prior art, the present invention provides a method, system, computer-readable storage medium and computer program product for quality inspection of head-up display windshields based on machine vision, with the aim of solving the problem of head-up display quality inspection of windshields.

[0006] To achieve the above objectives, according to one aspect of the present invention, a quality inspection method for head-up display windshields based on machine vision is provided, comprising:

[0007] The original image of the standard image projected by the head-up display system is obtained through the windshield to form the original virtual image of the projection, and the original image is processed into grayscale to obtain the corresponding grayscale image.

[0008] The ROI region, i.e. the virtual image region, is extracted from the grayscale image. The image with the extracted virtual image region is then subjected to double thresholding to segment the background region and the virtual image region, and to highlight the edge features of the virtual image, thus obtaining a thresholded image.

[0009] The system measures various physical parameters of the thresholded image and compares the measurement results with the corresponding parameters of the standard image. If the difference is within the preset threshold range, the head-up display shows that the image quality of the windshield is qualified; otherwise, it is unqualified.

[0010] Furthermore, it also includes using a similarity measurement algorithm to measure the difference between the standard image and the original image to determine the specific defect type. The similarity expression formula is as follows:

[0011] S p (M,T)=|f(M p )-f(T p )|

[0012]

[0013] Among them, S T (M,T) represents the global similarity between the standard image M and the original image T. The smaller the value, the higher the overall image quality of the windshield. p (M,T) represents the local similarity between the p-th block of the standard image M and the original image T. The smaller the value, the higher the local imaging quality of the windshield. The number of blocks in both the standard image M and the original image T is P, where p = 1 to P. p f(T) is the feature vector of the p-th block of the standard image M. p ) is the feature vector of the p-th block of the original image T;

[0014] The method for determining a specific quality category is as follows:

[0015]

[0016] Where a is an empirical value, S p (M,T)[A] represents all S p (M,T), S p (M,T)[I] represents any S p (M,T).

[0017] Furthermore, the feature vector of the original image T is directly composed of the physical parameters of the thresholded image obtained by measurement; the feature vector of the standard image M is composed of the parameters corresponding to the physical parameters of the original image T.

[0018] Furthermore, before extracting the ROI region from the grayscale image, a sliding window process is applied to the grayscale image:

[0019] For each pixel (x, y) in the grayscale image, select the median value (median(x, y)) within its neighborhood. ′ ,y ′ )) as the new value I of this pixel filtered (x,y).

[0020] Furthermore, the head-up display for windshield quality inspection is characterized by comprising a projection background, an industrial camera, a tooling bracket, a head-up display device, a memory, a processor, and a computer program stored in the memory;

[0021] Tooling brackets are used to secure windshields, industrial cameras, and head-up displays to simulate the layout, position, and angle of a car cabin.

[0022] The head-up display is used to output the test pattern to the windshield, thereby forming a virtual image on the projection background.

[0023] Industrial cameras are used to capture projected virtual images;

[0024] The processor executes the computer program to process the projected virtual image according to the quality inspection method as described in any of the preceding claims, thereby detecting the quality of the head-up display windshield.

[0025] According to another aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the quality inspection method as described in any of the preceding claims.

[0026] According to another aspect of the invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the quality inspection method as described in any of the preceding claims.

[0027] In summary, the technical solutions conceived in this invention, compared with the prior art, can achieve the following beneficial effects:

[0028] 1. This invention utilizes a machine vision-based virtual image detection and measurement method, enabling the detection system to determine windshield quality issues through acquired image data. This solves the problem that traditional measurement methods, which rely on the synergy of structured light scanning and laser thickness measurement, cannot efficiently assess windshield quality. This integrated approach improves measurement stability and efficiency while significantly reducing the overall cost of the detection system.

[0029] 2. To further determine the specific type of head-up display (HUD) quality problem in the windshield, this invention proposes an improved similarity algorithm based on the characteristics of HUD images. This algorithm can calculate the similarity between the actual test image (i.e., the original image) and the theoretical template image (i.e., the standard image) captured by the industrial camera, and can accurately determine the quality type of the output image. It can both study and analyze the overall features of the image and perform comparative calculations of local features, thus providing a better evaluation of the image quality of the windshield HUD.

[0030] 3. This improved similarity algorithm can be implemented either by independently acquiring feature vectors or by directly using feature vectors composed of measured physical parameters of the virtual image. It can be widely applied to the quality detection of windshield head-up displays under different experimental conditions. Attached Figure Description

[0031] Figure 1 This is the overall system measurement flowchart.

[0032] Figure 2 This is a schematic diagram of a windshield quality inspection system.

[0033] Figure 3 This is a flowchart of the similarity measurement algorithm. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0035] This invention proposes a machine vision-based measurement method and system for quality inspection of windshields in head-up display (HUD) systems. The system combines a HUD device and an industrial camera to perform quality inspection of target workpieces by simulating the actual environment of a car cabin. Specifically, the system is as follows: Figure 2 As shown, the head-up display device 5 outputs a test pattern, which is deformed by two reflections from an internal freeform mirror and an external curved windshield 2. An industrial camera 3 acquires high-precision data of the projected virtual image, and the quality of the windshield 2 is determined by analyzing the deformation of the projected virtual image. Before measurement, the system inputs the test pattern and calibrates the industrial camera 3. Then, it acquires the image data of the projected virtual image and processes it using an efficient algorithm applied by the data processing unit, ultimately outputting a high-precision quality assessment model. This method significantly improves the reliability and accuracy of product quality assessment.

[0036] Before the experiment, the windshield 2, industrial camera 3, and head-up display 5 were integrated onto the tooling bracket 4 according to the layout, position, and angle of the simulated car cockpit. The projection background 1 was placed directly in front of the windshield 2 according to the virtual image distance. Before the system was started, the calibration and position correction of the industrial camera 3 were completed.

[0037] The present invention preferably provides a quality inspection method for head-up display windshields based on machine vision, as shown in the appendix. Figure 1 The flowchart shown includes the following steps:

[0038] The industrial camera 3 acquires the original image of the virtual image projected by the head-up display device 5, and converts the image to grayscale to obtain the corresponding grayscale image.

[0039] Median filtering is applied to grayscale images to eliminate noise, light sources, and outliers, thereby improving image quality.

[0040] The ROI region (virtual image region) is extracted from the virtual image image. The virtual image region contains the defect information to be detected. The background region and the virtual image region are segmented by a thresholding method.

[0041] Double thresholding is applied to the image to obtain a defect image with clear features, highlighting the edge features of the virtual image and providing a basis for subsequent analysis.

[0042] For the preprocessed image, accurately measure the physical parameters related to the virtual image (such as size, degree of deformation, etc., which can be used to judge the image quality) to obtain the virtual image quality parameters.

[0043] For thresholded images containing defects and quality parameter information, a similarity measurement algorithm is used for measurement, or new quality parameters are directly detected and a similarity measurement algorithm is used for measurement, and a final quality assessment model is generated.

[0044] Optionally, the projection background plate 1 is installed directly in front of the windshield 2 to receive the virtual image projection and serve as a reference plate for auxiliary measurement. The windshield 2 is the target workpiece to be inspected and is also part of the optical path of the head-up display system. The industrial camera 3 is used to acquire the virtual image data of the head-up display system and is mounted on the fixture bracket 4. The fixture bracket 4 is assembled from aluminum profiles and is used to fix the industrial camera 3, the windshield 2, and the head-up display device 5 to simulate the layout, position, and angle of a car cabin.

[0045] Optionally, the head-up display 5 serves as the image source, responsible for projecting the selected test pattern. The data processing unit is connected to the industrial camera 3 and the head-up display 5 to integrate, process, and fuse the virtual image data.

[0046] Optionally, during camera calibration, Zhang Zhengyou's camera calibration method is used, which involves calculating the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameter matrix can be represented as:

[0047]

[0048] Among them, f x and f y c represents the focal length. x and c y Indicates the coordinates of the image center.

[0049] Optionally, in step S20, during the grayscale conversion process, I is used. gray =0.299×I R +0.587×I G +0.114×I B Convert the original RGB image to a grayscale image, where I R I G I B These represent the pixel values ​​for the red, green, and blue channels, respectively.

[0050] Optionally, the specific method for performing median filtering on a grayscale image is to process the image using a sliding window. For each pixel (x, y), the median value of its neighborhood, median(I(x′, y′)), is selected as the new value I for that pixel. filtered (x,y), i.e., I filtered (x,y)=median(I(x ′ ,y ′ )).

[0051] Optionally, ROI extraction uses a threshold-based image segmentation method. For example, the Otsu method can be used to automatically select the optimal threshold.

[0052] Alternatively, the method for performing double thresholding on the image can be to extract edges using the Canny edge detection algorithm. The calculation steps of the Canny algorithm include: Gaussian filtering; calculating the gradient intensity. Non-maximum suppression eliminates non-edge pixels; dual thresholding identifies weak and strong edges. x It is the gradient in the x-direction, G y It is the gradient in the y-direction.

[0053] Optionally, during the measurement of virtual image parameters, the virtual image is first fitted with a surface using the least squares method to fit its three-dimensional surface. Then, the relevant physical parameters are measured based on the fitted three-dimensional surface. The physical parameters measured here can be histograms, Hu moments, SIFT features, VLAD, BRISK, SURF, etc., as detailed in the table below (not exhaustively listed due to the nature of the relevant features being conventional):

[0054]

[0055] Optionally, a similarity measurement algorithm can be used to measure the difference between the standard image and the original image. This formula can be expressed as:

[0056] S p (M,T)=|f(M p )-f(T p )|

[0057]

[0058] Among them, S T (M,T) represents the global similarity between the standard image M and the original image T. The smaller the value, the higher the HUD imaging quality. p (M,T) represents the local similarity between the p-th block of the standard image M and the original image T, where the number of blocks in both the standard image M and the original image T is P, p = 1 to P, and f(M p f(T) is the feature vector of the p-th block of the standard image M. p Let be the feature vector of the p-th block of the original image T. The feature vector can be calculated or detected separately, or it can be directly composed of previously measured physical parameters.

[0059] 1 <![CDATA[S T (M,T)=0]]> <![CDATA[S p (M,T)[A]=0]]> qualified 2 <![CDATA[S T (M,T)=0]]> <![CDATA[S p (M,T)[A]≠0]]> Offset 3 <![CDATA[S T (M,T)=0]]> <![CDATA[S p (M,T)[A]≠0]]> Rotation 4 <![CDATA[S T (M,T)≥a]]> <![CDATA[S p (M,T)[I]=0]]> distortion 5 <![CDATA[0<S T (M,T)<a]]> <![CDATA[S p (M,T)[A]≠0]]> Double image

[0060] 'a' is an empirical value, preferably 0.3.

[0061] Generate the model for the final quality assessment:

[0062]

[0063] Where a is an empirical value, S p (M,T)[A] represents all S p (M,T), S p (M,T)[I] represents any S p (M,T).

[0064] This invention proposes a machine vision-based windshield quality inspection system. By projecting virtual image data and combining it with advanced computer vision algorithms, it can detect the optical performance and geometry of the windshield in real time, accurately assessing its impact on the head-up display system. Compared with existing technologies, this invention can not only detect external defects in the windshield but also effectively assess its potential impact on the vehicle's driver assistance system, thus providing a more comprehensive and accurate quality control solution.

[0065] This system combines a head-up display (HUD) and an industrial camera to simulate the actual environment of a car cabin for quality inspection of target workpieces. Specifically, the HUD outputs a test pattern, which deforms due to two reflections from an internal freeform mirror and an external curved windshield. The industrial camera then acquires high-precision data of the projected virtual image, analyzing its deformation to determine the windshield's quality. Before measurement, the system inputs the test pattern and calibrates the camera. It then acquires the image data of the projected virtual image and processes it using efficient algorithms in a data processing unit, ultimately outputting a high-precision quality assessment model, significantly improving the reliability and accuracy of product quality evaluation.

[0066] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A quality inspection method for head-up display windshields based on machine vision, characterized in that, include: The original image of the standard image projected by the head-up display system is obtained through the windshield to form the original virtual image of the projection, and the original image is processed into grayscale to obtain the corresponding grayscale image. The ROI region, i.e. the virtual image region, is extracted from the grayscale image. The image with the extracted virtual image region is then subjected to double thresholding to segment the background region and the virtual image region, and to highlight the edge features of the virtual image, thus obtaining a thresholded image. Measure various physical parameters of the thresholded image, compare the measurement results with the corresponding parameters of the standard image, and if the difference is within the preset threshold range, the head-up display shows that the image quality of the windshield is qualified; otherwise, it is unqualified. It also includes using a similarity measurement algorithm to measure the difference between the standard image and the original image to determine the specific defect type. The similarity expression formula is as follows: in, The global similarity between the standard image M and the original image T is denoted by , and the smaller the value, the higher the overall image quality of the windshield. Let be the local similarity of the p-th block between the standard image M and the original image T. The smaller the value, the higher the local imaging quality of the windshield. The number of blocks in both the standard image M and the original image T is P, where p = 1 to P. Let be the feature vector of the p-th block of the standard image M. This is the feature vector of the p-th block of the original image T; The method for determining a specific quality category is as follows: Where 'a' is an empirical value. [A] indicates all. , [I] represents any one ; Before extracting the ROI region from the grayscale image, a sliding window process is applied to the grayscale image: For each pixel (x, y) in the grayscale image, select the median value of its neighborhood. As the new value of this pixel .

2. The quality inspection method for head-up display windshields based on machine vision according to claim 1, characterized in that, The feature vector of the original image T is directly composed of the physical parameters of the thresholded image obtained by measurement; the feature vector of the standard image M is composed of the parameters corresponding to the physical parameters of the original image T.

3. A machine vision-based head-up display windshield quality inspection system, used for head-up display quality inspection of windshield (2), characterized in that, Includes a projection backdrop (1), an industrial camera (3), a tooling bracket (4), a head-up display device (5), a memory, a processor, and a computer program stored in the memory; The tooling bracket (4) is used to fix the windshield (2), industrial camera (3) and head-up display (5) to simulate the layout and angle of the car cabin; The head-up display (5) is used to output the test pattern to the windshield (2), thereby forming a virtual image on the projection background (1); An industrial camera (3) is used to acquire projected virtual images; The processor executes the computer program to process the projected virtual image according to the quality detection method of any one of claims 1 to 2, thereby detecting the quality of the head-up display windshield.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the quality inspection method according to any one of claims 1 to 2.

5. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the quality inspection method according to any one of claims 1 to 2.

Images