Machine vision-based screen splash test method and device and storage medium

Abstract

The invention provides a splash screen testing method based on machine vision. The splash screen testing method comprises the following steps: reading an image and cutting out a region of interest; carrying out preprocessing on the clipped image; calculating the difference between two adjacent frames of images to generate a difference chart; performing binarization processing on the difference image according to a threshold value, and extracting a change region; counting the number of pixels of which the pixel values are greater than a set threshold value in the change area, calculating the ratio of the number of the pixels to the total number of the pixels of the image, if the ratio is greater than the set threshold value, judging that a splash screen phenomenon occurs on the screen, and recording bounding box information of a splash screen area. According to the invention, the image information of the continuous frames of the screen is obtained through the camera, the pixel value difference of the two frames of images is automatically calculated, and whether the splash screen phenomenon occurs on the screen is judged according to whether significant brightness difference exists between the two adjacent frames of images; compared with a traditional manual identification method, the method can remarkably improve the efficiency and accuracy of the splash screen test, and provides a reliable basis for screen quality evaluation.

Description

Technical Field

[0001] The present invention relates to the field of screen detection technology, and in particular to a screen flash test method, device and storage medium based on machine vision. Background Art

[0002] With the continuous development of automotive electronic equipment, the popularization and application of instrument panel display technology has become an important part of modern automobile design. The stability and accuracy of instrument display directly affect driving safety and user experience. In order to ensure the reliability of automotive instruments during long-term use, screen aging testing has become an indispensable link.

[0003] Traditional screen aging test methods mainly rely on manual observation and simple test tools, which is not only inefficient, but also the results are easily affected by subjective factors, making it difficult to achieve high accuracy and consistency. In addition, with the continuous advancement of automotive electronics technology, display technology is also constantly evolving, facing a variety of problems such as screen noise and screen flicker, which put forward higher requirements for test methods.

[0004] In view of this, there is an urgent need for an automated screen flash testing method with real-time acquisition, data processing and result analysis capabilities to comprehensively monitor and analyze the automotive instrument screen in order to quickly and accurately identify various display anomalies, thereby providing a reliable quality assessment tool for automotive manufacturers and suppliers. Summary of the invention

[0005] The present invention proposes a screen flash test method, device and storage medium based on machine vision, which solves the problems that the screen aging test method in the prior art mainly relies on manual observation and simple test tools, is inefficient, and the results are easily affected by subjective factors, making it difficult to achieve high precision and consistency.

[0006] The technical solution of the present invention is achieved in this way:

[0007] A first aspect of the present invention provides a screen flash test method based on machine vision, comprising the following steps:

[0008] Read the image and crop the region of interest;

[0009] Preprocess the cropped image;

[0010] Calculate the difference between two adjacent frames of images and generate a difference map;

[0011] Binarize the difference map according to the threshold value and extract the changed area;

[0012] The number of pixels in the change area whose pixel values ​​are greater than the set threshold is counted, and the ratio of the number of pixels greater than the set threshold to the total number of pixels in the image is calculated. If the ratio is greater than the set threshold, it is determined that the screen flickers and the bounding box information of the flickering area is recorded.

[0013] Specifically, the method for reading an image is:

[0014] Update INI files as needed through the host computer;

[0015] Read INI files in real time to identify task requirements;

[0016] Start the CCD camera thread according to task requirements and continuously capture screen images;

[0017] Generate multiple queues to store images with different backlight colors respectively;

[0018] Create a dictionary with the key being the color information and the value being the image corresponding to the color;

[0019] Start the detection process and read the images in the queue according to the color index.

[0020] Furthermore, the system switches the backlight color of the screen to be tested according to the INI file, and turns off the screen 2S before each backlight color switch, and then turns on the screen after the backlight color switch.

[0021] Specifically, after reading the image, the image is masked according to a preset color threshold, the contour information of the target area is extracted through the mask, the perimeter of the contour is calculated, and the contour is approximated according to the perimeter to generate a polygon, and the area of ​​the polygon is calculated; it is determined whether the area of ​​the polygon exceeds the set threshold and whether the number of vertices of the polygon is 4. If both conditions are met, the polygon is the area of ​​interest.

[0022] Specifically, preprocessing the cropped image includes: converting the cropped image into a grayscale image, and performing Gaussian blur processing on the grayscale image.

[0023] Specifically, the method for generating a difference map is as follows:

[0024] Calculate the absolute difference of each pixel in two adjacent frames of images. The formula is as follows:

[0025] d(x,y)=|I_1(x,y)-I_2(x,y)|

[0026] Where d(x,y) represents the absolute difference of the pixels at the position (x,y) of the two frames, and I_1(x,y) and I_2(x,y) represent the pixel values ​​of the two frames at (x,y) respectively.

[0027] Create a new image, draw pixels in the new image according to d(x,y), and get a difference map. The pixel value at each position in the difference map corresponds to the pixel difference at that position between the two frames of images.

[0028] Specifically, the method for extracting the changed area is as follows: compare the pixel value of each position in the difference map with a preset value, if the pixel value is greater than the preset value, set the pixel value to 255, otherwise set the pixel value to 0, and obtain a binary image.

[0029] Specifically, after extracting the changed area, the OpenCV function is used to obtain the connected component information of the image, and the connected component information includes the number of components, pixel labels, statistical information and center points; each connected component is looped through, the area of ​​the connected area is extracted according to the statistical information, and the connected components with an area greater than the set threshold are screened out, and the bounding box coordinates and size of the connected component are recorded in the bounding box list.

[0030] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the screen flash test method when executing the computer program.

[0031] A third aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the screen flash test method are implemented.

[0032] Compared with the prior art, the beneficial effects of the present invention are: the present invention obtains image information of continuous frames of the screen through a camera, automatically calculates the difference in pixel values ​​between two frames of images, and determines whether the screen flickers according to whether there is a significant difference in brightness between the two adjacent frames of images; compared with the traditional method of manual identification, the present invention can significantly improve the efficiency and accuracy of screen flicker testing, and provide a reliable basis for screen quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a schematic diagram of the present invention;

[0035] Figure 2 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Reference Figure 1 The first aspect of the present invention provides a screen flash test method based on machine vision, comprising the following steps:

[0038] S1, updating the INI file as needed through the host computer, wherein the INI file mainly includes the relationship between status, color and quantity;

[0039] S2, the system will read the INI file cyclically to determine the changes in the requirements of the detection task in real time, ensure that the system can respond in time under different conditions, and flexibly adjust the detection strategy to meet the changing experimental requirements;

[0040] S3, start the CCD camera thread according to the task requirements, and continuously capture the screen image (in this embodiment, the XW800 industrial camera is used to capture the image of the automobile instrument displaying the specified test color in the test cabinet to ensure the accuracy of image acquisition);

[0041] In this embodiment, the host computer updates the INI file as needed to ensure that the system can be flexibly configured according to different experimental requirements, thereby improving the adaptability of the experiment. The transformation strategy of the INI file is as follows: Figure 2 As shown in the figure, in order to ensure that there is no frame leakage, the CCD recognition program uses an independent thread to collect images, which makes it impossible for the program to match the color parameters in the INI file in real time, so this strategy is adopted. This strategy requires the cooperation of the host computer software. The state parameter is set to OFF 2s before each backlight color switch, and then set to ON after the switch (restarting the CCD image acquisition thread).

[0042] S4, because the processing time of one frame of image is longer than the acquisition time of one frame of image, in order to improve efficiency, this embodiment uses multiple queues to store images of different backlight colors respectively;

[0043] S5, in order to facilitate the management and retrieval of image data, a dictionary is established, the key is the color information, and the value is the queue image corresponding to the color, so as to quickly find and access image data of different colors;

[0044] S6, start the detection process and read the image in the queue according to the color index;

[0045] S7, performing mask processing on the image according to a preset color threshold, and extracting contour information of the target area through the mask;

[0046] S8, calculating the perimeter of the contour, determining an accuracy parameter according to the perimeter, approximating the contour to generate a polygon, and calculating the area of ​​the polygon;

[0047] S9, judging whether the area of ​​the polygon exceeds the set threshold and whether the number of vertices of the polygon is 4. If both conditions are met, the polygon is confirmed to be a qualified rectangle (the rectangle can be considered as the screen display border. Other situations may occur during the detection process, such as partial area of ​​the screen with bad pixels and black screen, etc. Such situations need to be handled separately); and the judgment result is output in a table form for the host computer to read;

[0048] Since manufacturers have different requirements for screen detection, the screen flash detection in this embodiment can be performed according to specific needs without having to perform the detection every time.

[0049] S10, reading the image and cutting out the region of interest, retaining only the image data in the rectangular region;

[0050] S11, converting the cropped image into a grayscale image to reduce the computational complexity of subsequent processing;

[0051] S12, applies Gaussian blur with a convolution kernel size of (5,5) to the grayscale image to reduce the noise in the image;

[0052] S13, generating a difference map by calculating the absolute difference of pixel values ​​between two adjacent frame images frame1 and frame2, and performing binarization processing on the difference map to extract the changed area;

[0053] Calculate the absolute difference of each pixel in two adjacent frames of images. The formula is as follows:

[0054] d(x,y)=|I_1(x,y)-I_2(x,y)|

[0055] Where d(x,y) represents the absolute difference of the pixels at the position (x,y) of the two frames, and I_1(x,y) and I_2(x,y) represent the pixel values ​​of the two frames at (x,y) respectively.

[0056] Create a new image, draw pixels in the new image according to d(x,y), and get a difference map. The pixel value at each position in the difference map corresponds to the pixel difference at that position between the two frames of images.

[0057] The pixel value at each position in the difference map is compared with the preset value. If the pixel value is greater than the preset value, the pixel value is set to 255 (ie, white); otherwise, the pixel value is set to 0 (ie, black) to obtain a binary image.

[0058] S14, performing connected component analysis, using OpenCV functions to obtain connected component information of the image, wherein the connected component information includes the number of components, pixel labels, statistical information, and center points;

[0059] The OpenCV function will traverse each pixel in the binary image, and when it encounters an unlabeled foreground pixel, it will start a new component labeling; the pixels of the entire connected area will be assigned the same label, and then the number of components, pixel labels, statistical information and center points corresponding to the label will be counted and returned; the statistical information includes the starting coordinates, length and width of the circumscribed rectangle of each connected area and the number of pixels in the connected area;

[0060] S15, looping through each connected component, extracting the area of ​​the connected region according to the statistical information, and screening out the connected components whose area is greater than the set threshold, and recording the bounding box coordinates and size of the connected component into the bounding box list;

[0061] S16, counting the number of pixels in the change area whose pixel values ​​are greater than a set threshold and the total number of pixels in the image;

[0062] S17, calculating the ratio of the number of pixels greater than a set threshold to the total number of pixels in the image, if the ratio is greater than the set threshold, it is determined that the screen has a screen flicker phenomenon, and the bounding box information of the screen flicker area is recorded; if the ratio is less than or equal to the set threshold, it is determined that the screen has a pseudo screen flicker phenomenon, and the bounding box information of the pseudo screen flicker area is recorded;

[0063] S18, storing the splash screen image as video information, and drawing the bounding box information recorded in S15 into the video.

[0064] In this embodiment, all the set thresholds can be modified through user input to adapt to different types of screen detection requirements and improve the adaptability and accuracy of the system.

[0065] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the screen flash test method when executing the computer program.

[0066] A third aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the screen flash test method are implemented.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A screen flash test method based on machine vision, characterized in that: The following steps are involved: Read the image and crop the region of interest; Preprocess the cropped image; Calculate the difference between two adjacent frames of images and generate a difference map; Binarize the difference map according to the threshold value and extract the changed area; The number of pixels in the change area whose pixel values ​​are greater than the set threshold is counted, and the ratio of the number of pixels greater than the set threshold to the total number of pixels in the image is calculated. If the ratio is greater than the set threshold, it is determined that the screen flickers and the bounding box information of the flickering area is recorded.

2. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: The method to read the image is: Update INI files as needed through the host computer; Read INI files in real time to identify task requirements; Start the CCD camera thread according to task requirements and continuously capture screen images; Generate multiple queues to store images with different backlight colors respectively; Create a dictionary with the key being the color information and the value being the image corresponding to the color; Start the detection process and read the images in the queue according to the color index.

3. A screen flash test method based on machine vision as claimed in claim 2, characterized in that: The system switches the backlight color of the screen to be tested according to the INI file, and turns off the screen 2S before each backlight color switch, and then turns on the screen after the backlight color switch.

4. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: After reading the image, the image is masked according to the preset color threshold, the contour information of the target area is extracted through the mask, the perimeter of the contour is calculated, and the contour is approximated according to the perimeter to generate a polygon, and the area of ​​the polygon is calculated; it is determined whether the area of ​​the polygon exceeds the set threshold and whether the number of vertices of the polygon is 4. If both conditions are met, the polygon is the area of ​​interest.

5. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: The preprocessing of the cropped image includes: converting the cropped image into a grayscale image, and performing Gaussian blur processing on the grayscale image.

6. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: The method to generate the difference map is as follows: Calculate the absolute difference of each pixel in two adjacent frames of images. The formula is as follows: d(x,y)=|I_1(x,y)-I_2(x,y)| Where d(x,y) represents the absolute difference of the pixels at the position (x,y) of the two frames, and I_1(x,y) and I_2(x,y) represent the pixel values ​​of the two frames at (x,y) respectively. Create a new image, draw pixels in the new image according to d(x,y), and get a difference map. The pixel value at each position in the difference map corresponds to the pixel difference at that position between the two frames of images.

7. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: The method for extracting the changed area is as follows: compare the pixel value of each position in the difference map with the preset value, if the pixel value is greater than the preset value, set the pixel value to 255, otherwise set the pixel value to 0, and obtain a binary image.

8. A screen flash test method based on machine vision as claimed in claim 1, characterized in that: After extracting the changed area, the OpenCV function is used to obtain the connected component information of the image, which includes the number of components, pixel labels, statistical information and center points; each connected component is looped through, the area of ​​the connected area is extracted according to the statistical information, and the connected components with an area greater than the set threshold are screened out, and the bounding box coordinates and size of the connected component are recorded in the bounding box list.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the screen flash test method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the screen flash test method according to any one of claims 1 to 8 are implemented.

Images