Automatic optical detection device and control method thereof

Abstract

The present application provides an automatic optical inspection device and a control method thereof. The automatic optical inspection device is optically calibrated to obtain an image of a car display screen frame, and a single-channel image of the display screen frame is determined, thereby determining a smoothing matrix of the display screen frame image. According to the smoothing matrix of the display screen frame image, a gradient matrix of the display screen frame image is determined, thereby determining a filling image of the display screen frame. Contour fitting is performed on the filling image of the display screen frame to determine an optimal coefficient vector for contour fitting, thereby determining a corner point positioning coordinate sequence of the display screen frame, thereby determining a pixel spacing sequence of the display screen frame image, and thereby determining an actual size of the display screen frame. If the actual size of the car display screen frame is inconsistent with a standard car display screen frame size, it is marked as a car display screen frame with a size defect, thereby improving the accuracy of detecting size defects of the display screen frame.

Description

Technical Field

[0001] The present application relates to the technical field of detection equipment, and more specifically, to an automatic optical detection equipment and a control method thereof. Background Art

[0002] Testing equipment is a type of tool and instrument specially designed to measure, monitor and analyze various physical, chemical or biological parameters to obtain accurate data and information. These devices are widely used in various fields, including industrial manufacturing, healthcare, environmental monitoring, scientific research and safety. They are used to ensure product quality, monitor process variables, diagnose diseases, control production processes, protect the environment and improve production efficiency. Whether it is temperature sensors, medical imaging equipment, chemical analysis instruments or machine vision systems, testing equipment plays a key role in modern society and promotes the continuous progress of science and engineering.

[0003] Automatic optical inspection equipment can be used to identify defects in the frame of a car's display screen during automobile manufacturing. The equipment uses a high-resolution camera and image processing algorithms to capture, analyze and determine problems on the display screen frame. The existing technology uses automatic optical inspection to directly inspect the frame of a car's display screen, and does not consider the impact of related influencing factors on the current pixel points, resulting in the accuracy of the car's display screen frame size defect detection being too low. Summary of the invention

[0004] Based on this, it is necessary to address the above technical problems. The present application provides an automatic optical inspection device and a control method thereof for solving the problem of low accuracy in detecting size defects of automobile display frames.

[0005] In order to solve the above technical problems, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a control method for an automatic optical inspection device, comprising:

[0007] After starting the automatic optical inspection device to inspect the frame of the automobile display screen, optical calibration is performed on the automatic optical inspection device to obtain an image of the frame of the automobile display screen;

[0008] Performing single-channel conversion on the display screen frame image to obtain a display screen frame single-channel image, performing convolution on the display screen frame single-channel image, and then determining a display screen frame image smoothing matrix;

[0009] Determine the display screen frame image gradient matrix according to the display screen frame image smoothing matrix, and then determine the filling image of the display screen frame, perform contour fitting on the filling image of the display screen frame to determine the optimal coefficient vector of contour fitting, and determine the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector;

[0010] Determine a corner point spacing sequence of the display screen frame image according to the corner point positioning coordinate sequence of the display screen frame, and determine the actual size of the display screen frame according to the corner point spacing sequence of the display screen frame image;

[0011] If the actual size of the display screen frame is different from the standard size of a car display screen frame, the display screen frame is marked as a car display screen frame with a size defect.

[0012] In some embodiments, optical calibration of the automatic optical inspection device specifically includes:

[0013] Collect multiple initial automobile display screen frame images;

[0014] Determine the pixel pitch and actual distance in each initial vehicle display frame image;

[0015] Determine the transition coefficient of the frame image of the automobile display screen according to the pixel pitch and the actual distance, and then determine the conversion coefficients of all initial frame images of the automobile display screen;

[0016] The automatic optical detection equipment is calibrated by the conversion coefficient, and the frame image of the automobile display screen is recaptured.

[0017] In some embodiments, determining the display screen frame image gradient matrix specifically includes:

[0018] Determining moving smoothing windows at different positions in the display screen frame image smoothing matrix according to a preset smoothing window;

[0019] For each moving smoothing window, determining the regional balance value and local spatial discreteness of the moving smoothing window;

[0020] Determine the gradient threshold of the central pixel point of the moving smoothing window according to the regional equalization value and the local spatial discreteness;

[0021] Determine the gradient characteristic value of the moving smooth window according to the gradient threshold and the pixel value of the central pixel point of the moving smooth window;

[0022] The display screen frame image gradient matrix is ​​determined according to the determined gradient characteristic values ​​of each moving smoothing window.

[0023] In some embodiments, determining the gradient threshold of the central pixel point of the moving smoothing window specifically includes:

[0024] Get the smooth pixel value f of the center pixel point of the moving smooth window with coordinates (i, j) (i，j) ;

[0025] Obtaining the regional equilibrium value η of the moving smoothing window;

[0026] Obtain the local spatial discreteness ζ of the moving smoothing window;

[0027] According to the smooth pixel value f of the central pixel point of the moving smooth window with coordinates (i, j) (i，j) , the regional equalization value η of the moving smooth window, and the local spatial discreteness ζ of the moving smooth window determine the gradient threshold of the central pixel point of the moving smooth window, wherein the gradient threshold is determined by the following formula:

[0028]

[0029] Among them, T (i，j) represents the gradient threshold of the central pixel point (i, j) in the moving smoothing window, and k represents the deviation coefficient of the gradient threshold.

[0030] In some embodiments, determining a corner point positioning coordinate sequence of a display screen frame specifically includes:

[0031] Determine a profile fitting curve according to the optimal coefficient vector;

[0032] The candidate corner point coordinates are determined according to the contour fitting curve, and then the corner point positioning coordinate sequence of the display screen frame is determined.

[0033] In some embodiments, determining the corner point positioning coordinate sequence of the display screen frame specifically includes:

[0034] Determine initial filling pixel points according to the contour fitting curve;

[0035] Determine the curvature value of the contour fitting curve according to the initial filling pixel points, and then determine the candidate corner point coordinate sequence;

[0036] Determine a local area mean value sequence according to the initial filling pixel points, and then determine a local area coordinate sequence;

[0037] A corner point positioning coordinate sequence is determined according to the local area coordinate sequence and the candidate corner point coordinate sequence.

[0038] In some embodiments, determining the actual size of the display screen frame specifically includes:

[0039] Get all corner point spacings in the corner point spacing sequence of the display screen frame image;

[0040] Obtain the conversion coefficient between the pixel pitch of the frame image of the automobile display screen and the actual distance;

[0041] The actual size corresponding to each corner point spacing is determined according to the conversion coefficient and all corner point spacings in the corner point spacing sequence, and then the actual size of the automobile display screen frame is determined.

[0042] In a second aspect, the present application provides an automatic optical inspection device, which includes a size defect detection unit, and the size defect detection unit includes:

[0043] The automobile display screen frame image acquisition module is used to start the automatic optical detection equipment to detect the automobile display screen frame, and then perform optical calibration on the automatic optical detection equipment to obtain the automobile display screen frame image;

[0044] A display screen frame image smoothing matrix determination module is used to perform single-channel conversion on the display screen frame image to obtain a display screen frame single-channel image, perform convolution on the display screen frame single-channel image, and then determine the display screen frame image smoothing matrix;

[0045] A corner point positioning coordinate sequence determination module is used to determine a display screen frame image gradient matrix according to the display screen frame image smoothing matrix, thereby determining a filling image of the display screen frame, performing contour fitting on the filling image of the display screen frame to determine an optimal coefficient vector for contour fitting, and determining a corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector;

[0046] A display screen frame actual size determination module, used to determine a corner point spacing sequence of a display screen frame image according to a corner point positioning coordinate sequence of the display screen frame, and determine the display screen frame actual size according to the corner point spacing sequence of the display screen frame image;

[0047] The automobile display screen frame marking module is used to mark the automobile display screen frame as an automobile display screen frame with size defects if the actual size of the automobile display screen frame is different from the standard automobile display screen frame size.

[0048] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the control method of the automatic optical inspection device described above.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the control method of the automatic optical inspection device described above is implemented.

[0050] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0051] In the automatic optical inspection device and control method provided by the present application, after the automatic optical inspection device is started to inspect the automobile display screen frame, the automatic optical inspection device is optically calibrated to obtain the automobile display screen frame image; the display screen frame image is converted into a single channel to obtain a display screen frame single channel image, and the display screen frame single channel image is convolved to determine the display screen frame image smoothing matrix; the display screen frame image gradient matrix is ​​determined according to the display screen frame image smoothing matrix, and then the filling image of the display screen frame is determined, and the contour fitting is performed on the filling image of the display screen frame to determine the optimal coefficient vector of the contour fitting, and the corner point positioning coordinate sequence of the display screen frame is determined according to the optimal coefficient vector; the corner point spacing sequence of the display screen frame image is determined according to the corner point positioning coordinate sequence of the display screen frame, and the actual size of the display screen frame is determined according to the corner point spacing sequence of the display screen frame image; if the actual size of the display screen frame is different from the standard automobile display screen frame size, then The automobile display screen frame is marked as an automobile display screen frame with size defects. The scheme calibrates the automatic optical detection equipment through a checkerboard calibration plate, which can ensure the accuracy of the optical measurement results. Secondly, the smooth pixel value of the central pixel point in the moving convolution window is determined, and the influence of the pixel values ​​of all pixels in the moving convolution window on the central pixel point is considered, which can better retain the edges and details in the display screen frame image. Then, the gradient threshold of the central pixel point of the moving smoothing window is determined, and the influence of the regional balance value and local spatial discreteness of the moving smoothing window on the central pixel point is considered, which can clearly segment the display screen frame from the background of the display screen frame. Finally, by determining the contour fitting curve of the filling image of the display screen frame, the feature information of the display screen frame image can be made more obvious, thereby determining and comparing the actual size of the automobile display screen frame, and detecting the automobile display screen frame with size defects. Compared with the prior art, the accuracy of the automobile display screen frame size defect detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is an exemplary flow chart of a control method of an automatic optical inspection device according to some embodiments of the present application;

[0053] Figure 2 is a schematic diagram of exemplary hardware and / or software of a size defect detection unit according to some embodiments of the present application;

[0054] Figure 3 It is a structural schematic diagram of a computer device for implementing a control method for an automatic optical inspection device according to some embodiments of the present application.

[0055] Figure 4 It is a schematic diagram of a car display screen frame according to some embodiments of the present application. DETAILED DESCRIPTION

[0056] The core of the present application is to start the automatic optical detection device to detect the automobile display screen frame, then optically calibrate the automatic optical detection device to obtain the automobile display screen frame image; perform single-channel conversion on the display screen frame image to obtain the display screen frame single-channel image, perform convolution on the display screen frame single-channel image, and then determine the display screen frame image smoothing matrix; determine the display screen frame image gradient matrix according to the display screen frame image smoothing matrix, and then determine the filling image of the display screen frame; perform contour fitting on the filling image of the display screen frame to determine the optimal coefficient vector of contour fitting, and determine the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector; determine the corner point spacing sequence of the display screen frame image according to the corner point positioning coordinate sequence of the display screen frame, and determine the actual size of the display screen frame according to the corner point spacing sequence of the display screen frame image; if the actual size of the automobile display screen frame is different from the standard automobile display screen frame size, the automobile display screen frame is The display frame is marked as a car display frame with size defects. The scheme calibrates the automatic optical inspection equipment through a checkerboard calibration plate to ensure the accuracy of the optical measurement results. Secondly, the smooth pixel value of the central pixel in the moving convolution window is determined, and the influence of the pixel values ​​of all pixels in the moving convolution window on the central pixel is considered, which can better retain the edges and details in the display frame image. Then, the gradient threshold of the central pixel of the moving smoothing window is determined, and the influence of the regional balance value and local spatial discreteness of the moving smoothing window on the central pixel is considered, which can clearly segment the display frame from the background of the display frame. Finally, by determining the contour fitting curve of the filling image of the display frame, the feature information of the display frame image can be made more obvious, thereby determining and comparing the actual size of the car display frame, and detecting the car display frame with size defects. Compared with the prior art, the accuracy of size defect detection of the car display frame is improved.

[0057] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a control method of an automatic optical inspection device according to some embodiments of the present application. The control method 100 of the automatic optical inspection device mainly includes the following steps:

[0058] In step 101, after starting the automatic optical inspection device to inspect the frame of the automobile display screen, the automatic optical inspection device is optically calibrated to obtain an image of the frame of the automobile display screen.

[0059] In some embodiments, after the automatic optical inspection device is started to inspect the frame of the automobile display screen, the automatic optical inspection device is optically calibrated to obtain the image of the frame of the automobile display screen, which can be specifically obtained in the following manner, namely:

[0060] Collect multiple initial automobile display screen frame images;

[0061] Determine the pixel pitch and actual distance in each initial vehicle display frame image;

[0062] Determine the transition coefficient of the frame image of the automobile display screen according to the pixel pitch and the actual distance, and then determine the conversion coefficients of all initial frame images of the automobile display screen;

[0063] The automatic optical detection equipment is calibrated by the conversion coefficient, and the frame image of the automobile display screen is recaptured.

[0064] In specific implementation, after starting the automatic optical inspection device, the specifications of the checkerboard calibration plate can be set to 5mm×5mm, the checkerboard calibration plate can be placed under the car display frame, the automatic optical inspection device is fixed, and the position of the current automatic optical inspection device is used as the starting point and the position of the car display frame is used as the center to rotate one circle, and an image is taken every 12°. The image is used as the initial car display frame image, and a total of 30 different initial car display frame images are taken. OpenCV is used to extract the pixel coordinates corresponding to the intersection points of the grid on the checkerboard calibration plate in each initial car display frame image, and the pixel coordinates are calculated according to the distance formula between two points to obtain the initial car display frame image. The pixel distance in the display screen frame image corresponding to the intersection of the grid on the chessboard calibration plate is measured by using a vernier caliper. The actual distance between the automobile display screen frame image and the pixel points corresponding to the intersection of the grid on the chessboard calibration plate is measured. The actual distance is divided by the pixel distance to obtain the transition coefficient of the initial automobile display screen frame image. The average value of the transition coefficients of 30 initial automobile display screen frame images is used as the conversion coefficient. The conversion coefficient is set to the optical calibration parameter value of the automatic optical inspection equipment. The automatic optical inspection equipment is used again to capture the image of the automobile display screen frame, and the image is used as the automobile display screen frame image. It should be noted that the actual meaning of the conversion coefficient is the actual distance corresponding to adjacent pixel points.

[0065] In step 102, single-channel conversion is performed on the display screen frame image to obtain a display screen frame single-channel image, and convolution is performed on the display screen frame single-channel image to determine a display screen frame image smoothing matrix.

[0066] In some embodiments, the display screen frame image is converted into a single channel to obtain a display screen frame single channel image in the following manner, namely:

[0067] Determining the channel intensity coefficient of each color channel in the display screen frame image;

[0068] The single channel value of each pixel in the display screen frame image is determined according to each channel intensity coefficient, thereby obtaining the display screen frame single channel image.

[0069] In a specific implementation, the channel intensity coefficient of the red channel of the display screen frame image is set to 0.299, the channel intensity coefficient of the green channel is set to 0.587, and the channel intensity coefficient of the blue channel is set to 0.144. The color channel corresponding to a pixel point of the display screen frame image is converted into a single channel according to the channel intensity coefficient to obtain a single channel value. For example, R, G, and B respectively represent the three color channels of the red channel, green channel, and blue channel of the display screen frame image, then the single channel value I=0.299*R+0.587*G+0.114*B, and the pixel value of the pixel point corresponding to the single channel value in the display screen frame image is replaced according to each single channel value, and the replaced display screen frame image is used as the display screen frame single channel image.

[0070] In some embodiments, the display screen frame single-channel image is convolved to determine the display screen frame image smoothing matrix in the following manner, namely:

[0071] Determining different moving convolution windows in the single-channel image of the display screen frame according to a preset convolution window;

[0072] For each moving convolution window, determining the intensity magnitude and the single channel mean in the moving convolution window;

[0073] Determine the absolute center discrete difference in the moving convolution window according to the single channel mean and the single channel value of the central pixel point in the moving convolution window;

[0074] Determine a smoothed pixel value of a central pixel point in the moving convolution window according to the intensity value and the absolute central discrete difference;

[0075] The display screen frame image smoothing matrix is ​​determined according to the smoothed pixel values ​​of the central pixels in each determined moving convolution window.

[0076] In the specific implementation, the unit adjacent pixel spacing is taken as the radius, with the current pixel as the center, and the four pixels above, below, left and right as the boundaries of the preset convolution window, that is: its matrix form is similar to the shape of a cross For each moving convolution window, all single-channel values ​​in the moving convolution window are sorted in descending order, the maximum value of the single-channel value in the moving convolution window is taken as the intensity value, the average value of all single-channel values ​​in the moving convolution window is calculated, and the average value of the single-channel values ​​in the moving convolution window is taken as the single-channel mean; the single-channel mean is subtracted from the single-channel value of the center pixel in the moving convolution window and the absolute value is taken, and the absolute value is taken as the absolute center discrete difference of the center pixel in the moving convolution window. It should be noted that the unit adjacent pixel spacing in this application refers to the distance between any adjacent pixels in the single-channel image of the display screen frame.

[0077] In some embodiments, determining different moving convolution windows in the single-channel image of the display screen frame according to the convolution window may be implemented by the following steps:

[0078] Determining a sliding order according to the convolution window;

[0079] During the sliding process, the shape of the convolution window is adjusted to determine the moving convolution windows at different positions in the single-channel image of the display screen frame.

[0080] In a specific implementation, the center point of the convolution window is made to correspond to the position of the first pixel point in the upper left corner of the single-channel image of the display screen frame, that is, the center pixel point of the convolution window is the first pixel point in the upper left corner of the single-channel image of the display screen frame, and the convolution window is slid in a sliding order from left to right and from top to bottom; during the sliding process, when the center pixel point in the convolution window corresponds to the pixel point in the single-channel image of the display screen frame, if the convolution window contains less than 4 pixels, the shape of the convolution window is adjusted, and the adjustment rule is to retain the neighboring pixels whose maximum distance from the center pixel point of the convolution window is the unit adjacent pixel spacing, and use the adjusted convolution window as the moving convolution window corresponding to the center pixel point of the current convolution window. For example, for the case where the first pixel point in the upper left corner of the single-channel image of the display screen frame meets the above adjustment requirements, the shape of the convolution window is adjusted, that is: The adjusted convolution window is used as the moving convolution window corresponding to the first pixel point in the upper left corner of the single-channel image of the display screen frame. When the central pixel point in the convolution window corresponds to the pixel point in the single-channel image of the display screen frame, if the convolution window contains 4 pixels, the convolution window is directly used as the moving convolution window corresponding to the central pixel point of the current convolution window. According to the above steps, the shape of the convolution window at different positions in the single-channel image of the display screen frame is adjusted, and the adjusted convolution window is used as the moving convolution window at different positions in the single-channel image of the display screen frame. It should be noted that the adjacent pixel points in this application refer to all pixel points adjacent to any pixel point of the single-channel image of the display screen frame.

[0081] In some embodiments, determining the smoothed pixel value of the central pixel point in the moving convolution window according to the intensity value and the absolute central discrete difference is implemented by the following steps:

[0082] Get the single-channel values ​​of all single-channel image pixels of the display screen frame in the moving convolution window;

[0083] Get the single channel value f of the pixel point of the single channel image of the display screen frame with the center pixel point coordinate (i, j) in the moving convolution window (i，j) ;

[0084] Obtain the absolute discrete difference C of the center of the moving convolution window;

[0085] Get the intensity value q of the moving convolution window;

[0086] According to the single-channel values ​​of all the single-channel image pixels of the display screen frame in the moving convolution window, the single-channel value f of the single-channel image pixel of the display screen frame whose central pixel coordinate is (i, j) in the moving convolution window (i，j) , the absolute discrete difference C of the center of the moving convolution window, and the intensity value of the moving convolution window determine the smoothed pixel value of the central pixel point in the moving convolution window, wherein the smoothed pixel value is determined by the following formula:

[0087]

[0088] Among them, g (i，j) represents the smoothed pixel value of the central pixel point in the moving convolution window with coordinates (i, j), R represents the total number of pixels of the display frame single-channel image contained in the moving convolution window, D represents the convolution window, represents the central absolute deviation value in the moving convolution window, (m, n)∈D represents the pixel point of the display frame single-channel image with coordinates (m, n) in the moving convolution window, and f (m，n) Represents a single-channel value with pixel coordinates (m, n) of the single-channel image of the display screen frame in the moving convolution window.

[0089] It should be noted that the center absolute discrete difference described in the present application can constrain the deviation of the central pixel point in the moving convolution window, and the intensity value can constrain the value range of the center absolute discrete difference.

[0090] In some embodiments, a display screen frame image smoothing matrix is ​​determined according to the smoothed pixel values ​​of the central pixels in each determined moving convolution window, wherein the display screen frame image smoothing matrix can be expressed as follows:

[0091]

[0092] Among them, G represents the display frame image smoothing matrix, G 11 Represents the smoothed pixel value of the first row and first column in the display frame image smoothing matrix, G 12 G represents the smoothed pixel value of the first row and second column in the display frame image smoothing matrix. Similarly, the smoothed pixel value of the display frame image smoothing matrix in the first row and nth column is represented as G 1n , G 21 represents the smoothed pixel value of the 2nd row and 1st column in the display frame image smoothing matrix. Similarly, the smoothed pixel value of the display frame image smoothing matrix in the mth row and 1st column is G m1 , the smoothed pixel value of the mth row and nth column is represented as G mn .

[0093] In step 103, a display screen frame image gradient matrix is ​​determined according to the display screen frame image smoothing matrix, a filling image of the display screen frame is determined according to the display screen frame image gradient matrix, contour fitting is performed on the filling image of the display screen frame to determine an optimal coefficient vector for contour fitting, and a corner point positioning coordinate sequence of the display screen frame is determined according to the optimal coefficient vector;

[0094] In some embodiments, determining the display screen frame image gradient matrix according to the display screen frame image smoothing matrix may be implemented by the following steps:

[0095] Determining moving smoothing windows at different positions in the display screen frame image smoothing matrix according to a preset smoothing window;

[0096] For each moving smoothing window, determining the regional balance value and local spatial discreteness of the moving smoothing window;

[0097] Determine the gradient threshold of the central pixel point of the moving smoothing window according to the regional equalization value and the local spatial discreteness;

[0098] Determine the gradient characteristic value of the moving smooth window according to the gradient threshold and the pixel value of the central pixel point of the moving smooth window;

[0099] The display screen frame image gradient matrix is ​​determined according to the determined gradient characteristic values ​​of each moving smoothing window.

[0100] In the specific implementation, the first smooth pixel point in the upper left corner of the smooth matrix is ​​taken as the starting point, and the spacing between three unit smooth pixels is taken as the length. The boundaries are extended to the right and downward to determine the boundaries, and a square smooth window with a size of 3×3 is obtained. Its matrix form is During the sliding process, for each moving smoothing window, the mean of all smoothed pixel values ​​in the moving smoothing window is calculated, and the mean is used as the regional balance value. The variance of all smoothed pixel values ​​in the moving smoothing window is calculated, and the value of the variance is used as the local spatial discreteness. It should be noted that the unit smoothing pixel point spacing in this application refers to the distance between any two adjacent smoothing pixel points in the smoothing matrix.

[0101] In some embodiments, determining the moving smoothing windows at different positions in the display screen frame image smoothing matrix according to the smoothing window can be implemented by the following steps:

[0102] Determining a sliding order according to the smoothing window;

[0103] During the sliding process, the shape of the smoothing window is adjusted to determine the moving smoothing windows at different positions in the display screen frame image smoothing matrix.

[0104] In a specific implementation, the center point of the smoothing window is made to correspond to the position of the first pixel point in the upper left corner of the smoothing matrix, that is, the center pixel point of the smoothing window is the first pixel point in the upper left corner of the smoothing matrix, and the smoothing window is slid in a sliding order from left to right and from top to bottom; during the sliding process, when the center pixel point of the smoothing window corresponds to the smoothing pixel point in the smoothing matrix, if the smoothing window contains less than 8 smoothing pixel points, the shape of the smoothing window is adjusted, and the adjustment rule is to retain the maximum distance from the center pixel point of the smoothing window to The neighboring smoothing pixel points with a unit smoothing pixel spacing are adjusted, and the adjusted smoothing window is used as the moving smoothing window corresponding to the central pixel point of the current smoothing window. For example, if the first smoothing pixel point in the upper left corner of the smoothing matrix meets the above adjustment requirements, the shape of the smoothing window is adjusted, that is: The adjusted smoothing window is used as the moving smoothing window corresponding to the first smoothing pixel point in the upper left corner of the smoothing matrix. When the center pixel point of the smoothing window corresponds to the smoothing pixel point in the smoothing matrix, if the smoothing window contains 8 smoothing pixels, the smoothing window is directly used as the moving smoothing window corresponding to the center pixel point of the current smoothing window. According to the above steps, the shapes of the smoothing windows at different positions in the smoothing matrix of the display screen frame image are adjusted to obtain the moving smoothing windows at different positions in the smoothing matrix of the display screen frame image. It should be noted that the adjacent smoothing pixels in the present application refer to all smoothing pixels adjacent to any smoothing pixel point in the smoothing matrix.

[0105] In some embodiments, determining the gradient threshold of the central pixel point of the moving smoothing window according to the regional equalization value and the local spatial discreteness is implemented by the following steps:

[0106] Get the smooth pixel value f of the center pixel point of the moving smooth window with coordinates (i, j) (i，j) ;

[0107] Obtaining the regional equilibrium value η of the moving smoothing window;

[0108] Obtain the local spatial discreteness ζ of the moving smoothing window;

[0109] According to the smooth pixel value f of the central pixel point of the moving smooth window with coordinates (i, j) (i，j) , the regional equalization value η of the moving smooth window, and the local spatial discreteness ζ of the moving smooth window determine the gradient threshold of the central pixel point of the moving smooth window, wherein the gradient threshold is determined by the following formula:

[0110]

[0111] Among them, T (i，j) represents the gradient threshold of the central pixel point (i, j) in the moving smoothing window, and k represents the deviation coefficient of the gradient threshold.

[0112] It should be noted that the regional balance value described in the present application is used to measure the balance degree of all smooth pixel points in the moving smooth window, and the local spatial discreteness can measure the deviation degree of all smooth pixel points in the moving smooth window.

[0113] In addition, it should be noted that the deviation coefficient of the gradient threshold in the present application needs to be determined by continuous adjustment according to the separation effect of the frame in the display screen frame image, and the adjustment method is to start from greater than 0 until a satisfactory separation effect is achieved.

[0114] In some embodiments, the gradient characteristic value of the moving smooth window can be determined according to the gradient threshold and the pixel value of the central pixel point of the moving smooth window by using the following formula:

[0115]

[0116] Among them, S ij represents the gradient eigenvalue of the moving smoothing window at the coordinate (i, j), T (i，j) represents the gradient threshold at coordinate (i, j) in the smoothing matrix, f (i，j) Indicates the smoothed pixel value of the center pixel point of the moving smoothing window with coordinates (i, j).

[0117] In specific implementation, if the gradient threshold with coordinates (i, j) in the smoothing matrix is ​​greater than the smoothed pixel value with coordinates (i, j) of the central pixel point of the moving smoothing window, the gradient eigenvalue of the moving smoothing window at coordinates (i, j) is set to 0; if the gradient threshold with coordinates (i, j) in the smoothing matrix is ​​less than or equal to the smoothed pixel value with coordinates (i, j) of the central pixel point of the moving smoothing window, the gradient eigenvalue of the moving smoothing window at coordinates (i, j) is set to 255.

[0118] In some embodiments, the display screen frame image gradient matrix is ​​determined according to the determined gradient feature values ​​of each moving smoothing window, wherein the display screen frame image gradient matrix can be expressed as follows:

[0119]

[0120] Among them, S represents the display screen frame image gradient matrix, S 11 represents the gradient eigenvalue of the first row and first column in the gradient matrix of the display frame image, S 12 It represents the gradient eigenvalue of the first row and second column in the gradient matrix of the display screen frame image. Similarly, the gradient eigenvalue of the first row and nth column in the gradient matrix of the display screen frame image is expressed as S 1n , S 21 It represents the gradient eigenvalue of the 2nd row and 1st column in the gradient matrix of the display screen frame image. Similarly, the gradient eigenvalue of the mth row and 1st column in the gradient matrix of the display screen frame image is expressed as S m1 , the gradient eigenvalue of the mth row and nth column in the gradient matrix of the display frame image is expressed as S mn .

[0121] In some embodiments, the filling image of the display screen frame may be determined according to the display screen frame image gradient matrix in the following manner:

[0122] An erosion operation and an expansion operation are performed on the gradient matrix of the display screen frame to obtain a morphological image of the display screen frame, and a filling operation is performed on the morphological image, and the morphological image after the filling operation is used as a filling image of the display screen frame.

[0123] It should be noted that the erosion operation, dilation operation and filling operation in the present application are all existing morphological technologies, among which the erosion operation can remove the fine edges of the contour of the display screen frame gradient image, the dilation operation can supplement the missing parts of the contour of the display screen frame gradient image, and the filling operation can remove the holes in the display screen frame gradient image.

[0124] In some embodiments, the contour fitting of the filling image of the display screen frame is performed to determine the optimal coefficient vector of the contour fitting in the following manner, namely:

[0125] Get the fill pixel coordinates of the fill image of the display screen frame;

[0126] According to the filling pixel coordinates, fitting the frame contour in the filling image of the display screen frame to determine a general polynomial for contour fitting;

[0127] Determine a matrix equation for profile fitting based on the general polynomial;

[0128] The optimal coefficient vector for profile fitting is determined according to the matrix equation.

[0129] In a specific implementation, according to the obtained filling pixels of the filling image of the display screen frame, the horizontal coordinate and the vertical coordinate of one of the filling pixels are randomly selected, and the general polynomial of the contour fitting is determined according to the obtained coordinates of the filling pixel, wherein the general polynomial can be determined by the following formula:

[0130]

[0131] Among them, x j ,y j Indicates the horizontal and vertical coordinates of the jth pixel of the filled image, a i (i=0,1,2,…,N) represents the general polynomial coefficient, which is a random number sequence, b j The hyperparameter representing the general polynomial corresponding to the j-th pixel point is a random number. N represents the order of the general polynomial. It should be noted that the order of the general polynomial in this application is adjusted according to the effect of contour fitting, and is usually an odd number greater than 0, for example: 3, 5, 7, 9.

[0132] In addition, it should be noted that the filling pixels of the filling image of the display screen frame in the present application are the pixel points where the frame is located on the filling image of the display screen frame.

[0133] In some embodiments, the matrix equation for contour fitting can be determined according to the general polynomial in the following manner, namely:

[0134] Get the coordinates of all the filled pixel points of the filled image of the display screen frame;

[0135] According to the general polynomial, all the filling pixels in the filling image of the display screen frame are fitted to determine the matrix equation of the contour fitting, wherein the matrix equation of the contour fitting is determined according to the following formula:

[0136] Y=Xθ+b

[0137] in, Represents the vertical coordinate vector of the filled pixel point, Represents the horizontal coordinate vector of the filled pixel point, The hyperparameter vector representing the matrix equation for the profile fit, The coefficient vector of the matrix equation for contour fitting, k represents the total number of filled pixels, and N represents the order of the general polynomial.

[0138] In the specific implementation, all the filling pixels in the filling image of the display screen frame are obtained, and all the filling pixels are fitted according to the general polynomial fitting method to obtain the matrix equation of the contour fitting. It should be noted that the elements in the hyperparameter vector and the coefficient vector are all randomly generated random numbers.

[0139] In some embodiments, the optimal coefficient vector for contour fitting may be determined according to the matrix equation in the following manner, namely:

[0140] Determine a loss function for contour fitting according to the matrix equation;

[0141] An optimal coefficient vector for contour fitting is determined according to the loss function of the contour fitting.

[0142] Using the principle of least squares method, the coordinates of all the filled pixel points in the filled image of the display screen frame are used to find the optimal solution of the coefficient vector θ of the matrix equation, so that the inner product of the hyperparameter vector of the matrix equation is minimized, that is: The value of is minimized, the function expression when the inner product of the hyperparameter vector is minimized is used as the loss function of the profile fitting, the derivative of the loss function with respect to the coefficient vector θ is solved, and the coefficient vector corresponding to zero when the derivative is zero is used as the optimal coefficient vector of the profile fitting. In specific implementation, the loss function of the profile fitting can be determined by the following formula:

[0143] L(θ)=[Y-Xθ] T [Y-Xθ]

[0144] Where L(θ) represents the loss function of the profile fitting with respect to the coefficient vector θ, Y=Xθ+b represents the matrix equation, Represents the horizontal coordinate vector of the filled pixel point, The hyperparameter vector representing the matrix equation for the profile fit, The coefficient vector of the matrix equation for contour fitting, k represents the total number of filled pixels, and N represents the order of the general polynomial.

[0145] It should be noted that the MATLAB toolbox is used to solve the derivative of the coefficient vector in this application. In practice, other methods of solving the derivative can also be used, which are not specifically limited here.

[0146] In some embodiments, the following method can be used to determine the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector, namely:

[0147] Determine a profile fitting curve according to the optimal coefficient vector;

[0148] The candidate corner point coordinates are determined according to the contour fitting curve, and then the corner point positioning coordinate sequence of the display screen frame is determined.

[0149] In a specific implementation, the number of components of the optimal coefficient vector is used as the highest order of the general polynomial, and a contour fitting curve is determined according to the highest order and the optimal coefficient vector, wherein the contour curve is determined by the following formula:

[0150]

[0151] Among them, x and y represent the horizontal and vertical coordinates of any filled pixel points on the contour fitting curve, and M represents the highest order of the horizontal coordinate of any filled pixel points on the contour fitting curve. Represents the optimal coefficient vector. It should be noted that the contour fitting curve in this application is a closed curve.

[0152] In some embodiments, the following method can be used to determine the candidate corner point coordinates according to the contour fitting curve, and then determine the corner point positioning coordinate sequence of the display screen frame, namely:

[0153] Determine initial filling pixel points according to the contour fitting curve;

[0154] Determine the curvature value of the contour fitting curve according to the initial filling pixel points, and then determine the candidate corner point coordinate sequence;

[0155] Determine a local area mean value sequence according to the initial filling pixel points, and then determine a local area coordinate sequence;

[0156] Determine a corner point positioning coordinate sequence according to the local area coordinate sequence and the candidate corner point coordinate sequence;

[0157] In specific implementation, for example, OpenCV is used to select a filling pixel point on the contour fitting curve corresponding to the lower left corner of the display screen frame as the initial filling pixel point. The car display screen frame can be as follows Figure 4As shown in the figure, the black dot in the lower left corner indicates the position of the initial filling pixel point; starting from the initial filling pixel point, in a clockwise direction, the curvature values ​​of all adjacent filling pixel points on the contour fitting curve are calculated in turn, and the curvature values ​​of all adjacent filling pixel points are sorted in the order of calculation, and the sorted sequence is used as the curvature sequence. Starting from the first curvature value in the curvature sequence, the curvature values ​​in the curvature sequence are compared in turn. If the previous curvature value is greater than the next curvature value, the coordinates of the filling pixel point corresponding to the previous curvature value are used as candidate coordinates, until all the curvature values ​​in the curvature sequence are compared, all ordered candidate coordinates are obtained, and all ordered candidate coordinates are used as candidate coordinate sequence; starting from the initial filling pixel point, 9 adjacent filling pixel points on the contour fitting curve are selected in turn as local areas and the average of the pixel values ​​of the adjacent 9 filling pixel points is calculated, and the average is used as the local area average of the initial filling pixel point, and in a clockwise direction, the local area averages of all the filling pixel points on the contour fitting curve are calculated in turn to obtain all ordered local areas. Domain mean, take the ordered local area mean as the local area mean sequence, take the initial filling pixel point as the starting point, and compare the pixel values ​​of all the filling pixels on the contour fitting curve with the corresponding local area mean in the local area mean sequence in a clockwise direction. If the pixel value of the filling pixel is greater than the corresponding local area mean in the local area mean sequence, take the coordinates of the filling pixel point as the local area coordinate, until the pixel values ​​of all the filling pixels on the contour fitting curve are compared, all the ordered local area coordinates are obtained, and all the ordered local area coordinates are taken as the local area coordinate sequence; take the candidate coordinates in the candidate coordinate sequence as the reference, start from the first candidate coordinate in the candidate coordinate sequence, and compare all the local area coordinates in the local area coordinate sequence in turn. If the candidate coordinate is the same as the local area coordinate, take the candidate coordinate as the corner point positioning coordinate, until all the candidate coordinates in the candidate coordinate sequence are traversed, all the ordered corner point positioning coordinates are obtained, and all the ordered corner point positioning coordinates are taken as the corner point positioning coordinate sequence of the display screen frame;

[0158] In addition, it should be noted that the curvature of the filled pixel point on the contour fitting curve in the present application represents the local feature of the contour fitting curve of the display screen frame at that point, and describes the degree of curvature of the contour fitting curve at that point.

[0159] In step 104, a corner point spacing sequence of the display screen frame image is determined according to the corner point positioning coordinate sequence of the display screen frame, and an actual size of the display screen frame is determined according to the corner point spacing sequence of the display screen frame image.

[0160] In some examples, the following method can be used to determine the corner point spacing sequence of the display screen frame image according to the corner point positioning coordinate sequence of the display screen frame, namely:

[0161] Obtaining the corner point positioning coordinates in the corner point positioning coordinate sequence of the display screen frame;

[0162] The corner point spacing sequence of the display screen frame image is determined according to the corner point positioning coordinates.

[0163] In a specific implementation, the first corner point positioning coordinate in the corner point positioning coordinate sequence of the display screen frame is taken as the starting point, and according to the distance formula between two points in the prior art, the distance between adjacent corner point positioning coordinates is taken as the corner point spacing, and the adjacent corner point positioning coordinates in the corner point positioning coordinate sequence of the display screen frame are calculated in sequence to obtain three groups of ordered corner point spacings, and the three groups of ordered corner point spacings are taken as the initial corner point spacing sequence, and the distance between the last corner point positioning coordinate and the first corner point positioning coordinate in the corner point positioning coordinate sequence of the display screen frame is calculated to obtain the last group of corner point spacings, and the last group of corner point spacings is added to the back of the last group of corner point spacings in the initial corner point spacing sequence to obtain four groups of ordered corner point spacings, and the four groups of ordered corner point spacings are taken as the corner point spacing sequence of the display screen frame image;

[0164] In some examples, the actual size of the display screen frame may be determined according to the corner point spacing sequence of the display screen frame image in the following manner, namely:

[0165] Get all corner point spacings in the corner point spacing sequence of the display screen frame image;

[0166] Obtain the conversion coefficient between the pixel pitch of the frame image of the automobile display screen and the actual distance;

[0167] The actual size corresponding to each corner point spacing is determined according to the conversion coefficient and all corner point spacings in the corner point spacing sequence, and then the actual size of the display screen frame is determined.

[0168] In specific implementation, for each corner point spacing in the corner point spacing sequence of the display screen frame image, a value obtained by multiplying each corner point spacing by a conversion coefficient is used as the actual size corresponding to the corner point spacing, and the actual sizes corresponding to all corner point spacings are used as the actual size of the display screen frame.

[0169] In step 105 , if the actual size of the display screen frame is different from the standard size of the automobile display screen frame, the display screen frame is marked as an automobile display screen frame with size defects.

[0170] In some examples, if the actual size of the display screen frame is different from the standard size of a car display screen frame, the display screen frame is marked as a car display screen frame with a size defect.

[0171] In specific implementation, if the actual size of the automobile display frame detected by the automatic optical inspection equipment is inconsistent with the standard automobile display frame size, the automobile display frame will be marked as a automobile display frame with size defects. It should be noted that the standard automobile display frame size in this application can be obtained by measuring using a high-precision caliper.

[0172] In addition, in another aspect of the present application, in some embodiments, the present application provides an automatic optical inspection device, the device comprising a size defect inspection unit, referring to Figure 2 , which is a schematic diagram of exemplary hardware and / or software of a size defect detection unit according to some embodiments of the present application, the size defect detection unit 200 includes: an automobile display screen frame image acquisition module 201, a display screen frame image smoothing matrix determination module 202, a corner point positioning coordinate sequence determination module 203, a display screen frame actual size determination module 204 and an automobile display screen frame marking module 205, which are respectively described as follows:

[0173] The automobile display screen frame image acquisition module 201 in the present application is mainly used to start the automatic optical detection device to detect the automobile display screen frame, and then perform optical calibration on the automatic optical detection device to obtain the automobile display screen frame image;

[0174] The display screen frame image smoothing matrix determination module 202, the display screen frame image smoothing matrix determination module 202 in the present application is mainly used to perform single-channel conversion on the display screen frame image to obtain a display screen frame single-channel image, perform convolution on the display screen frame single-channel image, and then determine the display screen frame image smoothing matrix;

[0175] Corner point positioning coordinate sequence determination module 203, the display screen frame corner point positioning coordinate sequence determination module 203 in the present application is mainly used to determine the display screen frame image gradient matrix according to the display screen frame image smoothing matrix, and then determine the filling image of the display screen frame, perform contour fitting on the filling image of the display screen frame to determine the optimal coefficient vector of contour fitting, and determine the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector;

[0176] The actual size determination module 204 of the display screen frame is mainly used to determine the corner point spacing sequence of the display screen frame image according to the corner point positioning coordinate sequence of the display screen frame, and determine the actual size of the display screen frame according to the corner point spacing sequence of the display screen frame image;

[0177] The automobile display screen frame marking module 205, in the present application, the automobile display screen frame size defect marking module 205 is mainly used to mark the display screen frame as an automobile display screen frame with size defects if the actual size of the display screen frame is different from the standard automobile display screen frame size.

[0178] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the control method of the above-mentioned automatic optical inspection device.

[0179] In some embodiments, reference Figure 3 , which is a schematic diagram of the structure of a computer device according to some embodiments of the present application using the control method of the automatic optical detection device. The control method of the automatic optical detection device in the above embodiment can be Figure 3 The computer device shown in the figure is implemented, and the computer device includes at least one processor 301, a communication bus 302, a memory 303 and at least one communication interface 304.

[0180] The processor 301 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the control method of the automatic optical inspection device in the present application.

[0181] Communication bus 302 may include a pathway for transmitting information between the above-mentioned components.

[0182] The memory 303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0183] The memory 303 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the program code stored in the memory 303. The program code may include one or more software modules. The determination of the display screen frame image smoothing matrix in the above embodiment can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0184] The communication interface 304 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0185] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0186] The above-mentioned computer device can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0187] In addition, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the control method of the above-mentioned automatic optical inspection device is implemented.

[0188] In summary, in the automatic optical inspection device and control method thereof disclosed in the embodiment of the present application, after the automatic optical inspection device is started to inspect the frame of the automobile display screen, the automatic optical inspection device is optically calibrated to obtain an image of the frame of the automobile display screen; the display screen frame image is converted into a single channel to obtain a single channel image of the display screen frame, and the single channel image of the display screen frame is convolved to determine a smoothing matrix of the display screen frame image; the display screen frame image gradient matrix is ​​determined according to the display screen frame image smoothing matrix, and the filling image of the display screen frame is determined; the filling image of the display screen frame is contour fitted to determine the optimal coefficient vector of contour fitting, and the corner point positioning coordinate sequence of the display screen frame is determined according to the optimal coefficient vector; the corner point spacing sequence of the display screen frame image is determined according to the corner point positioning coordinate sequence of the display screen frame, and the actual size of the display screen frame is determined according to the corner point spacing sequence of the display screen frame image; if the actual size of the automobile display screen frame is different from the standard automobile display screen frame size If the size of the automobile display screen frame is different, the automobile display screen frame is marked as an automobile display screen frame with size defects. The scheme calibrates the automatic optical detection equipment through a checkerboard calibration plate, which can ensure the accuracy of the optical measurement results. Secondly, the smooth pixel value of the central pixel point in the moving convolution window is determined, and the influence of the pixel values ​​of all pixels in the moving convolution window on the central pixel point is considered, which can better retain the edges and details in the display screen frame image. Then, the gradient threshold of the central pixel point of the moving smoothing window is determined, and the influence of the regional balance value and local spatial discreteness of the moving smoothing window on the central pixel point is considered, which can clearly segment the display screen frame from the background of the display screen frame. Finally, by determining the contour fitting curve of the filling image of the display screen frame, the feature information of the display screen frame image can be made more obvious, thereby determining and comparing the actual size of the automobile display screen frame, and detecting the automobile display screen frame with size defects. Compared with the prior art, the accuracy of the automobile display screen frame size defect detection is improved.

[0189] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0190] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A control method for an automatic optical inspection device, characterized in that: The steps include: After starting the automatic optical inspection device to inspect the frame of the automobile display screen, optical calibration is performed on the automatic optical inspection device to obtain an image of the frame of the automobile display screen; Performing single-channel conversion on the display screen frame image to obtain a display screen frame single-channel image, performing convolution on the display screen frame single-channel image, and then determining a display screen frame image smoothing matrix; Determine the display screen frame image gradient matrix according to the display screen frame image smoothing matrix, and then determine the filling image of the display screen frame, perform contour fitting on the filling image of the display screen frame to determine the optimal coefficient vector of contour fitting, and determine the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector; Determine a corner point spacing sequence of the display screen frame image according to the corner point positioning coordinate sequence of the display screen frame, and determine the actual size of the display screen frame according to the corner point spacing sequence of the display screen frame image; If the actual size of the display screen frame is different from the standard size of the automobile display screen frame, the display screen frame is marked as an automobile display screen frame with size defects; Among them, optical calibration of the automatic optical inspection equipment to obtain the frame image of the automobile display screen specifically includes: Collect multiple initial automobile display screen frame images; Determine the pixel pitch and actual distance in each initial vehicle display frame image; Determine the transition coefficient of the frame image of the automobile display screen according to the pixel pitch and the actual distance, and then determine the conversion coefficients of all initial frame images of the automobile display screen; The automatic optical detection equipment is calibrated by the conversion coefficient, and the frame image of the automobile display screen is recaptured; Wherein, determining the display screen frame image gradient matrix according to the display screen frame image smoothing matrix specifically includes: Determining moving smoothing windows at different positions in the display screen frame image smoothing matrix according to a preset smoothing window; For each moving smoothing window, determining the regional balance value and local spatial discreteness of the moving smoothing window; Determine the gradient threshold of the central pixel point of the moving smoothing window according to the regional equalization value and the local spatial discreteness; Determine the gradient characteristic value of the moving smooth window according to the gradient threshold and the pixel value of the central pixel point of the moving smooth window; Determine the display screen frame image gradient matrix according to the determined gradient eigenvalues ​​of each moving smoothing window; Wherein, determining the corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector specifically includes: Determine a profile fitting curve according to the optimal coefficient vector; Determine the coordinates of the candidate corner points according to the contour fitting curve, and then determine the corner point positioning coordinate sequence of the display screen frame; The step of determining the candidate corner point coordinates according to the contour fitting curve and then determining the corner point positioning coordinate sequence of the display screen frame specifically includes: Determine initial filling pixel points according to the contour fitting curve; Determine the curvature value of the contour fitting curve according to the initial filling pixel points, and then determine the candidate corner point coordinate sequence; Determine a local area mean value sequence according to the initial filling pixel points, and then determine a local area coordinate sequence; Determine a corner point positioning coordinate sequence according to the local area coordinate sequence and the candidate corner point coordinate sequence; Wherein, determining the actual size of the display screen frame according to the corner point spacing sequence of the display screen frame image specifically includes: Get all corner point spacings in the corner point spacing sequence of the display screen frame image; Obtain the conversion coefficient between the pixel pitch of the frame image of the automobile display screen and the actual distance; The actual size corresponding to each corner point spacing is determined according to the conversion coefficient and all corner point spacings in the corner point spacing sequence, and then the actual size of the automobile display screen frame is determined.

2. The method according to claim 1, characterized in that Determining the gradient threshold of the central pixel point of the moving smoothing window according to the regional equalization value and the local spatial discreteness specifically includes: Get the smooth pixel value f of the center pixel point of the moving smooth window with coordinates (i, j) (i，j) ; Obtaining the regional equilibrium value η of the moving smoothing window; Obtain the local spatial discreteness ζ of the moving smoothing window; According to the smooth pixel value f of the center pixel point of the moving smooth window with coordinates (i, j) (ij) , the regional equalization value η of the moving smooth window and the local spatial discreteness ζ of the moving smooth window determine the gradient threshold of the central pixel point of the moving smooth window, wherein the gradient threshold is determined by the following formula: Among them, T (i，j) represents the gradient threshold of the central pixel point (i, j) in the moving smoothing window, and k represents the deviation coefficient of the gradient threshold.

3. An automatic optical inspection device, which is controlled by the method according to any one of claims 1 or 2, characterized in that: The device includes a size defect detection unit, and the size defect detection unit includes: The automobile display screen frame image acquisition module is used to start the automatic optical detection equipment to detect the automobile display screen frame, and then perform optical calibration on the automatic optical detection equipment to obtain the automobile display screen frame image; A display screen frame image smoothing matrix determination module is used to perform single-channel conversion on the display screen frame image to obtain a display screen frame single-channel image, perform convolution on the display screen frame single-channel image, and then determine the display screen frame image smoothing matrix; A corner point positioning coordinate sequence determination module is used to determine a display screen frame image gradient matrix according to the display screen frame image smoothing matrix, thereby determining a filling image of the display screen frame, performing contour fitting on the filling image of the display screen frame to determine an optimal coefficient vector for contour fitting, and determining a corner point positioning coordinate sequence of the display screen frame according to the optimal coefficient vector; A display screen frame actual size determination module, used to determine a corner point spacing sequence of a display screen frame image according to a corner point positioning coordinate sequence of the display screen frame, and determine the display screen frame actual size according to the corner point spacing sequence of the display screen frame image; The automobile display screen frame marking module is used to mark the display screen frame as an automobile display screen frame with size defects if the actual size of the display screen frame is different from the standard automobile display screen frame size.

4. A computer device, comprising a memory and a processor, wherein the memory stores a code, characterized in that: The processor is configured to obtain the code and execute the control method of the automatic optical inspection device according to any one of claims 1 or 2.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the control method of the automatic optical inspection device according to any one of claims 1 or 2 is implemented.

Images