An abnormal detection method, system and device for the side of a screen body

Abstract

The present application discloses an abnormal detection method, system and device for the side of a screen body, which is used to improve the efficiency of abnormal detection on the side of the screen body. The method of the present application includes: taking an initial image of the side of the display screen; determining the edge area of the initial image; splitting the edge area according to a preset splitting density to obtain a plurality of analysis areas; respectively connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; successively obtaining all the gray values of all the pixel points on the plurality of reference lines; traversing all the gray values according to a preset rule to determine the outermost edge points of the screen body; fitting the outermost edge points of the screen body to obtain an actual detection area; traversing all the gray values within the actual detection area according to the preset rule to determine the edge points of the abnormal area; and fitting the edge points of the abnormal area to obtain an abnormal area.

Description

Technical Field

[0001] This application relates to the field of screen detection, and in particular, to a method, system, and device for detecting abnormalities on the side of a screen body. Background Art

[0002] With the development of display technology, display screens are widely used in various electronic devices. However, during actual use, display screens may have problems on the side of the screen body due to external forces, aging, etc., such as cracks, deformations, or abnormal displays. If these problems are not detected and processed in a timely manner, they may affect the overall performance of the device and even cause serious damage.

[0003] In the prior art, display screen detection methods mainly focus on the central area of the screen, and relatively few detections are carried out on the side of the screen body, resulting in low efficiency in detecting abnormalities on the side of the screen body. Summary of the Invention

[0004] To solve the above technical problems, this application provides a method, system, and device for detecting abnormalities on the side of a screen body, which is used to improve the efficiency of detecting abnormalities on the side of the screen body.

[0005] The technical solutions provided in this application are described below:

[0006] In the first aspect of this application, a method for detecting abnormalities on the side of a screen body is provided, including:

[0007] Taking an initial image of the side of the display screen;

[0008] Determining the edge region of the initial image;

[0009] Splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions;

[0010] Connecting the midpoints of the two short sides of each of the analysis regions respectively to obtain a plurality of reference lines;

[0011] Obtaining all the gray values of all the pixel points on the plurality of reference lines one by one;

[0012] Traversing all the gray values according to a preset rule to determine the outermost edge points of the screen body;

[0013] Fitting the outermost edge points of the screen body to obtain the actual detection region;

[0014] Traversing all the gray values in the actual detection region according to the preset rule to determine the edge points of the abnormal region;

[0015] Fitting the edge points of the abnormal region to obtain the abnormal region.

[0016] Optionally, traversing all the grayscale values according to the preset rule to determine the outermost edge points of the screen body includes:

[0017] Traverse all the grayscale values in the positive direction, and calculate the first grayscale value difference between two adjacent grayscale values among all the grayscale values to obtain a first grayscale value difference set;

[0018] Obtain the grayscale value differences less than the first preset value in the first grayscale value difference set, and record the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target points to obtain a first positive target point set. The first preset value is used to determine whether the pixel points corresponding to two grayscale values satisfy the change state from dark to bright;

[0019] Obtain the grayscale value differences greater than the second preset value in the first grayscale value difference set, and record the pixel points corresponding to the grayscale value differences greater than the second preset value as the second positive target points to obtain a second positive target point set. The second preset value is used to determine whether the pixel points corresponding to two grayscale values satisfy the change state from bright to dark;

[0020] Traverse all the grayscale values in the reverse direction, and calculate the second grayscale value difference between two adjacent grayscale values among all the grayscale values to obtain a second grayscale value difference set;

[0021] Obtain the grayscale value differences less than the first preset value in the second grayscale value difference set, and record the pixel points corresponding to the grayscale value differences less than the first preset value as the first negative target points to obtain a first negative target point set;

[0022] Obtain the grayscale value differences greater than the second preset value in the second grayscale value difference set, and record the pixel points corresponding to the grayscale value differences greater than the second preset value as the second negative target points to obtain a second negative target point set;

[0023] Obtain the overlapping points of the first positive target point set and the second negative target point set to obtain a first overlapping point set;

[0024] Obtain the overlapping points of the second positive target point set and the first negative target point set to obtain a second overlapping point set;

[0025] Determine the outermost edge points of the screen body from the first overlapping point set and the second overlapping point set according to the preset edge area.

[0026] Optionally, traversing all the grayscale values in the actual detection area according to the preset rule to determine the edge points of the abnormal area includes:

[0027] Obtain the overlapping points of the first overlapping point set and the second overlapping point set within the actual detection area to obtain the edge points of the abnormal area.

[0028] Optionally, the fitting of the outermost edge points of the screen body to obtain the actual detection area includes:

[0029] Determine the coordinate values of the pixel points corresponding to the outermost edge points of the screen body on the initial image, and establish a least squares model according to the coordinate values;

[0030] Calculate the sum of the squared errors between the minimized predicted values and the observed values of the least squares model, and determine the slope and intercept of the fitted target line;

[0031] Determine the actual detection area on the initial image according to the slope and intercept.

[0032] Optionally, the splitting of the edge area according to the preset splitting density to obtain multiple analysis areas includes:

[0033] Obtain the minimum bounding rectangle of the initial image, and perform edge extraction according to the minimum bounding rectangle to determine the edge area;

[0034] Split the edge area according to the preset splitting density to obtain multiple analysis areas.

[0035] Optionally, the obtaining of the minimum bounding rectangle of the initial image and the performing of edge extraction according to the minimum bounding rectangle to determine the edge area includes:

[0036] Use the sobel operator to calculate the gradients of the initial image in the horizontal and vertical directions to obtain the gradient magnitude and gradient direction;

[0037] Scan the initial image according to the gradient magnitude and the gradient direction, and sequentially obtain and connect the pixel points that satisfy the gradient magnitude on the initial image to obtain the edge area.

[0038] Optionally, the fitting of the edge points of the abnormal area to obtain the abnormal area includes:

[0039] Analyze the height differences of adjacent overlapping points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image;

[0040] When the height difference is within the error range of the same level, determine that the adjacent overlapping points used to calculate the height difference belong to the same level;

[0041] Determine all levels within the actual detection area that contain the edge points of the abnormal area;

[0042] Connect the edge points of the abnormal areas at each level one by one to obtain the edge of the abnormal area;

[0043] Close the edge of the abnormal area to obtain the abnormal area.

[0044] The second aspect of the present application provides an abnormal detection system on the side of a screen body, including:

[0045] A shooting unit for shooting an initial image of the side of the display screen;

[0046] A first determination unit for determining the edge area of the initial image;

[0047] A splitting unit for splitting the edge area according to a preset splitting density to obtain a plurality of analysis areas;

[0048] A connection unit for connecting the midpoints of the two short sides of each of the analysis areas respectively to obtain a plurality of reference lines;

[0049] An acquisition unit for acquiring all gray values of all pixel points on the plurality of reference lines one by one;

[0050] A first traversal unit for traversing all the gray values according to a preset rule to determine the outermost edge points of the screen body;

[0051] A first fitting unit for fitting the outermost edge points of the screen body to obtain the actual detection area;

[0052] A second traversal unit for traversing all the gray values within the actual detection area according to the preset rule to determine the edge points of the abnormal area;

[0053] A second fitting unit for fitting the edge points of the abnormal area to obtain the abnormal area.

[0054] Optionally, the first traversal unit is specifically configured to:

[0055] Traverse all the gray values in the forward direction, and calculate the first gray value difference between two adjacent gray values among all the gray values to obtain a first gray value difference set;

[0056] Obtain the gray value differences less than a first preset value in the first gray value difference set, and record the pixel points corresponding to the gray value differences less than the first preset value as the first forward target point positions to obtain a first forward target point position set, where the first preset value is used to determine whether the pixel points corresponding to two gray values satisfy the change state from dark to bright;

[0057] Obtain the gray value differences greater than a second preset value in the first gray value difference set, and record the pixel points corresponding to the gray value differences greater than the second preset value as the second positive target points, to obtain a second positive target point set, where the second preset value is used to determine whether the pixel points corresponding to two gray values satisfy the change state from bright to dark;

[0058] Traverse all the gray values in reverse order, and calculate the second gray value differences between adjacent two gray values among all the gray values, to obtain a second gray value difference set;

[0059] Obtain the gray value differences less than the first preset value in the second gray value difference set, and record the pixel points corresponding to the gray value differences less than the first preset value as the first negative target points, to obtain a first negative target point set;

[0060] Obtain the gray value differences greater than the second preset value in the second gray value difference set, and record the pixel points corresponding to the gray value differences greater than the second preset value as the second negative target points, to obtain a second negative target point set;

[0061] Obtain the overlapping points of the first positive target point set and the second negative target point set, to obtain a first overlapping point set;

[0062] Obtain the overlapping points of the second positive target point set and the first negative target point set, to obtain a second overlapping point set;

[0063] Determine the outermost edge points of the screen body from the first overlapping point set and the second overlapping point set according to a preset edge area.

[0064] Optionally, the second traversing unit is specifically configured to:

[0065] Obtain the overlapping points of the first overlapping point set and the second overlapping point set within the actual detection area, to obtain the edge points of the abnormal area.

[0066] Optionally, the first fitting unit is specifically configured to:

[0067] Determine the coordinate values of the pixel points corresponding to the outermost edge points of the screen body on the initial image, and establish a least squares method model according to the coordinate values;

[0068] Calculate the sum of the squared errors between the minimized predicted values and the observed values of the least squares method model, and determine the slope and intercept of the fitting target straight line;

[0069] Determine the actual detection area on the initial image according to the slope and intercept.

[0070] Optionally, the splitting unit is specifically configured to:

[0071] Obtain the minimum bounding rectangle of the initial image, and perform edge extraction based on the minimum bounding rectangle to determine the edge region;

[0072] Split the edge region according to a preset splitting density to obtain a plurality of analysis regions.

[0073] Optionally, the splitting unit is further specifically configured to:

[0074] Use the sobel operator to calculate the gradients of the initial image in the horizontal and vertical directions to obtain the gradient magnitude and gradient direction;

[0075] Scan the initial image according to the gradient magnitude and the gradient direction, and sequentially obtain and connect the pixel points on the initial image that satisfy the gradient magnitude to obtain the edge region.

[0076] Optionally, the second fitting unit is specifically configured to:

[0077] Analyze the height differences of adjacent overlapping points one by one from left to right according to the positional relationship of the edge points of the abnormal region on the initial image;

[0078] When the height differences are within the error range of the same level, determine that the adjacent overlapping points used to calculate the height differences belong to the same level;

[0079] Determine all levels in the actual detection region that contain the edge points of the abnormal region;

[0080] Connect the edge points of the abnormal region on each level one by one to obtain the edge of the abnormal region;

[0081] Close the edge of the abnormal region to obtain the abnormal region.

[0082] A third aspect of the present application provides an abnormal detection device for the side of a screen body, and the device includes:

[0083] A processor, a memory, an input / output unit, and a bus;

[0084] The processor is connected to the memory, the input / output unit, and the bus;

[0085] The memory stores a program, and the processor calls the program to execute the method of the first aspect and any optional method in the first aspect.

[0086] A fourth aspect of the present application provides a computer-readable storage medium, and a program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the method of the first aspect and any optional method in the first aspect.

[0087] As can be seen from the above technical solutions, the present application has the following advantages:

[0088] Capture an initial image of the side of the display screen;

[0089] Determine the edge region of the initial image;

[0090] Split the edge region according to a preset splitting density to obtain a plurality of analysis regions;

[0091] Connect the midpoints of the two short sides of each of the analysis regions respectively to obtain a plurality of reference lines;

[0092] Obtain all the gray values of all the pixel points on the plurality of reference lines one by one;

[0093] Traverse all the gray values according to a preset rule to determine the outermost edge points of the screen body;

[0094] Fit the outermost edge points of the screen body to obtain the actual detection region;

[0095] Traverse all the gray values in the actual detection region according to the preset rule to determine the edge points of the abnormal region;

[0096] Fit the edge points of the abnormal region to obtain the abnormal region.

[0097] Obtain an initial image of the side of the screen and determine the edge region of the side of the screen in the obtained initial image. Split the edge region to determine the reference lines for refining the edge of the side of the screen in the initial image. Determine the actual detection region by traversing the gray values of the pixels on each reference line. Analyze the gray values of the pixels on the reference line again according to the actual detection region to obtain the edge points of the abnormal region. Fit and connect the abnormal points to obtain the abnormal region determined by all the abnormal points in the actual side region. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the technical solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0099] Figure 1 It is a schematic flowchart of an embodiment of the abnormal detection method for the side of the screen body in the present application;

[0100] Figure 2a It is a schematic flowchart of another embodiment of the first stage of the abnormal detection method for the side of the screen body in the present application;

[0101] Figure 2bAnother schematic flow diagram of the second stage of the abnormal detection method for the side of the screen body in this application;

[0102] Figure 2c Another schematic flow diagram of the third stage of the abnormal detection method for the side of the screen body in this application;

[0103] Figure 3 A schematic structural diagram of an embodiment of the abnormal detection system for the side of the screen body in this application;

[0104] Figure 4 A schematic structural diagram of an embodiment of the abnormal detection device for the side of the screen body in this application;

[0105] Figure 5 A schematic diagram of the initial image in the abnormal detection method for the side of the screen body in this application. Detailed implementation manners

[0106] It should be noted that an abnormal detection method for the side of the screen body provided in this application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone, a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of description, this application takes the terminal as the execution subject for example.

[0107] Next, the technical solutions in this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0108] Please refer to Figure 1 , this application first provides an embodiment of the abnormal detection method for the side of the screen body, and this embodiment includes:

[0109] S101. Shoot an initial image of the side of the display screen;

[0110] The initial image is an image of the four sides around the screen body collected by an industrial camera after the actual screen production is completed and the screen body is assembled around.

[0111] In an actual shooting environment, the lens parameters of the industrial camera used for shooting are fixed, so that the aperture focal length of the industrial camera is maintained in a state that meets the working environment of the shooting system. The parameters that need to be adjusted are the positional relationship between the platform and the camera, and the relative position between the point light source and the camera lens. Generally, the point light source is set on one side of the industrial camera lens. The above parameters are recorded in the terminal in actual situations and can directly control the camera through the terminal during actual operation, enabling the camera to automatically obtain and set the corresponding parameters after power-on. After determining the parameters of the industrial camera, the screen body is adsorbed by the adsorption device, and the four surrounding images of the side of the screen body are collected by the rotation of the robotic arm. These images are the initial images of the screen.

[0112] S102. Determine the edge region of the initial image;

[0113] The terminal determines the edge region of the data on the side of the screen in the initial image by calculating the gradient magnitude and gradient direction of each point in the initial image, and performs non-maximum suppression on this edge region.

[0114] After the terminal completes non-maximum suppression, it will trace the pixel points that meet the gray-scale amplitude change along the gradient direction. If the weak edge pixels are connected to the strong edge, they will be retained as real edges, otherwise they will be discarded, and finally the complete edge region is output.

[0115] Among them, a strong edge refers to an edge with obvious brightness changes in the image, usually corresponding to the contour of an object or the boundary line between an object and the background. A strong edge appears as a high gradient value in the gradient magnitude image, that is, the brightness changes rapidly. In the output of the edge detection algorithm, strong edges usually appear as bright lines or dark lines because they are very prominent in the image. Strong edges are very important for image understanding and object recognition because they usually mark the boundaries of objects.

[0116] A weak edge refers to an area with less significant brightness changes in the image, which may correspond to the subtle changes or textures on the surface of an object. A weak edge appears as a low gradient value in the gradient magnitude image, that is, the brightness changes slowly. In the output of the edge detection algorithm, weak edges may not be very obvious because their gradient values are low. Weak edges may contain useful information in some cases, such as texture analysis or pattern recognition, but in most edge detection tasks, they are usually regarded as noise.

[0117] The goal of the edge detection algorithm is to identify and highlight the strong edges in the image while suppressing weak edges and noise as much as possible. This is because strong edges usually contain important information about the image structure and object shape.

[0118] In practical applications, edge detection algorithms (such as Sobel, Canny, Laplacian, etc.) usually use some thresholds to distinguish strong edges from weak edges. Edges exceeding the threshold are considered strong edges, while edges below the threshold are regarded as weak edges or noise. By adjusting the threshold, the sensitivity of the algorithm to edges can be controlled to meet different application requirements.

[0119] After the terminal determines the real area, it generates a minimum bounding rectangle for the real area and expands the minimum bounding rectangle by a certain number of pixels according to a preset value to obtain a rectangular frame that includes all the screen side information. This rectangular frame is the edge area. After determining the edge area, the terminal will split the edge area according to the preset splitting density, and each of the split areas is an analysis area. All the analysis areas are integrated into multiple analysis areas.

[0120] It should be noted that before determining the edge area of the initial image, the initial image should be preprocessed. The specific preprocessing methods are as follows:

[0121] The purpose of preprocessing is to make the information carried on the initial image clearer, that is, to perform noise reduction operations on the initial image to filter out or even eliminate interference information.

[0122] The process of preprocessing the initial image includes at least two steps: mean filtering and Gaussian filtering. Among them, the processing method of mean filtering is to filter the initial image through a 3*3 mean filtering window to obtain a grayscale image corresponding to the initial image, and then use the Gaussian function as a filter to perform weighted averaging on the signals on the image to smooth the signals and reduce noise.

[0123] After the terminal obtains the grayscale image, it will perform Gaussian filtering on the grayscale image. Gaussian filtering is actually filtering the initial image after mean filtering by inputting a Gaussian convolution kernel. The Gaussian convolution kernel used in this embodiment is as follows:

[0124]

[0125] Among them, G(x,y) is the grayscale value of the grayscale image at the coordinate (x,y), that is, x and y are the grayscale values of the pixel points corresponding to the coordinate parameters of the image. is the standard deviation of the Gaussian distribution, which determines the smoothness of the filter. Most of the noise will be removed from the filtered image, and the main image features will be retained.

[0126] S103. Split the edge area according to the preset splitting density to obtain multiple analysis areas;

[0127] The terminal determines the edge region of the screen side data in the initial image by calculating the gradient magnitude and gradient direction of each point in the initial image, and performs non-maximum suppression on this edge region.

[0128] After the terminal completes non-maximum suppression, it will trace the pixel points that satisfy the gray-scale amplitude change along the gradient direction. If the weak edge pixels are connected to the strong edge, they are retained as real edges, otherwise they are discarded, and finally the complete edge region is output.

[0129] Among them, a strong edge refers to an edge with obvious brightness changes in the image, usually corresponding to the contour of an object or the boundary between an object and the background. A strong edge appears as a high gradient value in the gradient magnitude image, that is, the brightness changes rapidly. In the output of the edge detection algorithm, strong edges usually appear as bright lines or dark lines because they are very prominent in the image. Strong edges are very important for image understanding and object recognition because they usually mark the boundaries of objects.

[0130] A weak edge refers to an area with less significant brightness changes in the image, which may correspond to the subtle changes or textures on the surface of an object. A weak edge appears as a low gradient value in the gradient magnitude image, that is, the brightness changes slowly. In the output of the edge detection algorithm, weak edges may not be very obvious because their gradient values are low. Weak edges may contain useful information in some cases, such as texture analysis or pattern recognition, but in most edge detection tasks, they are usually regarded as noise.

[0131] The goal of the edge detection algorithm is to identify and highlight the strong edges in the image while suppressing the weak edges and noise as much as possible. This is because strong edges usually contain important information about the image structure and object shape.

[0132] In practical applications, edge detection algorithms (such as Sobel, Canny, Laplacian, etc.) usually use some thresholds to distinguish between strong edges and weak edges. Edges exceeding the threshold are considered strong edges, while edges below the threshold are regarded as weak edges or noise. By adjusting the threshold, the sensitivity of the algorithm to edges can be controlled to meet different application requirements.

[0133] After the terminal determines the real region, it generates the minimum bounding rectangle for this real region, and expands the minimum bounding rectangle by a certain number of pixels according to a preset value to obtain a rectangular frame containing all the screen side information. This rectangular frame is the edge region. After determining the edge region, the terminal will split the edge region according to the preset splitting density, and each split region is an analysis region.

[0134] S104. Connect the midpoints of the two short sides of each analysis region respectively to obtain multiple reference lines;

[0135] After the terminal determines the analysis regions, it will successively determine the midpoints of the short sides of these analysis regions. The length of the short side of the analysis region can be calculated by the actual length of the long side of the minimum bounding rectangle on the initial image and the splitting density. Since the minimum bounding rectangle is essentially a rectangle, all the analysis regions are also rectangles and each analysis region has the same size. Therefore, the reference lines obtained by connecting the midpoints of the short sides are evenly distributed on the initial image. The reference lines are used to represent the actual pixel distribution of the corresponding analysis regions, that is, pixel sampling is performed on the initial image at a fixed density uniformly. And the reference lines are obtained by connecting the midpoints of the upper and lower short sides of the analysis region. The number of reference lines is the same as the number of analysis regions, and one reference line represents the pixel distribution of one analysis region.

[0136] S105. Successively obtain all the gray values of all the pixel points on multiple reference lines;

[0137] In the above steps, the terminal will scan and traverse the data carried on the image more than once to determine the noise reduction situation of the image. In actual situations, the border obtained by directly scanning the image captured (i.e., the above-mentioned minimum bounding rectangle) is a rough estimated border that necessarily covers all the data on the screen sides.

[0138] S106. Traverse all the gray values according to the preset rules to determine the outermost edge points of the screen body;

[0139] Specifically, when traversing the gray values, the terminal will judge whether the pixel point is an edge point according to the difference of the gray values. When the difference of the gray values is greater than the preset value, it means that this point is already the outermost edge point of the screen body. In actual situations, the traversal of the gray values needs to go through at least one forward traversal and one reverse traversal to improve the reliability of the outermost edge points of the screen body.

[0140] S107. Fit the outermost edge points of the screen body to obtain the actual detection region;

[0141] After the terminal determines the outermost edge points of the screen body of all the analysis regions, the terminal will establish a least squares model based on these outermost edge points of the screen body to fit the actual positions of the upper and lower sides of the screen side in the initial image according to these outermost edge points of the screen body, and close the actual detection region according to the actual positions of the upper and lower sides.

[0142] S108. Traverse all the gray values within the actual detection region according to the preset rules to determine the edge points of the abnormal region;

[0143] The terminal in the foregoing step generates a new side edge frame of the screen body in the initial image, that is, the actual detection area. The actual detection area fits better with the minimum circumscribed rectangle in the foregoing step and the side edge to be detected in the actual initial image. And the terminal can exclude the interference of the coincidence points outside the screen edge area and the screen coincidence area on the subsequent anomaly detection according to the actual detection area.

[0144] S109. Fit the edge points of the abnormal area to obtain the abnormal area.

[0145] The fitting process of the abnormal area is similar to the fitting process of the actual detection area, that is, a least squares model is established through the edge points of the abnormal area to fit the actual abnormal area.

[0146] Please refer to Figure 2a 、 Figure 2b and Figure 2c , another embodiment of the anomaly detection method for the side of the screen body provided by the embodiment of the present application includes:

[0147] S201. Take an initial image of the side of the display screen;

[0148] Step S201 in this embodiment is similar to step S101 in the foregoing embodiment, and will not be elaborated here specifically.

[0149] S202. Use the sobel operator to calculate the gradients of the initial image in the horizontal and vertical directions to obtain the gradient amplitude and the gradient direction;

[0150] In actual situations, the position of the screen side on the initial image is not absolutely a frontal image, that is, there may be an inclination of the screen side on the initial image. Therefore, after determining the calculation method of the Gaussian convolution kernel, it is also necessary to calculate the actual position of the screen side on the initial image. Specifically, it is to calculate the gradient Gx in the horizontal direction of the screen side image in the initial image and the gradient Gy in the vertical direction of the screen side image in the initial image. Gx is the gradient component of the image in the horizontal direction (x direction), indicating the brightness change rate of the image in the horizontal direction; Gy is the gradient component of the image in the vertical direction (y direction), indicating the brightness change rate of the image in the vertical direction. For each pixel point (x, y) in the image, its neighborhood pixel values are multiplied by the corresponding weights in the Sobel operator and then summed to obtain the horizontal gradient Gx and the vertical gradient Gy of this point.

[0151] Among them, Gx and Gy can be calculated through the Sobel operator, and the specific calculation is carried out through the following formula:

[0152]

[0153]

[0154] Among them, G is the calculation formula for the gradient magnitude, and is the formula for the gradient direction.

[0155] S203. Scan the initial image according to the gradient magnitude and gradient direction, and obtain and connect pixel points on the initial image that satisfy the gradient magnitude one by one to obtain the edge region.

[0156] After determining the direction and gradient of the screen side image content in the initial image according to the Sobel operator, the terminal will scan the gradient magnitude of the image according to the gradient direction, and at the same time compare whether the gradient magnitude of the currently scanned pixel and its two neighboring pixels in the neighborhood satisfies the gradient magnitude G calculated according to the above formula. If not, the gradient magnitude is recorded as 0, that is, maximum suppression is performed on the gray gradient magnitude in the horizontal or vertical direction.

[0157] The terminal will start from the strong edge determined by non-maximum suppression and trace along the gradient direction. The strong edge is the actual edge of the screen side determined by the terminal according to the gradient magnitude during the initial image scanning. However, for gray images, there may be cases where the complete strong edge cannot be obtained due to the light in the shooting environment and the shooting direction. Therefore, the terminal needs to determine whether the subsequent edge not determined as a strong edge is connected to the current strong edge. If connected, the subsequent edge is retained as a real edge, otherwise it is discarded. After determining the complete edge, the terminal will expand the upper and lower edges of the complete edge by a certain number of pixels according to a preset value to ensure that all the actual screen side information can be included in the minimum bounding rectangle. At this time, the connected complete edges can determine the edge region of the initial image, and this edge region is the edge region of the initial image that contains all the screen side information.

[0158] Specifically, the process of the terminal determining the strong edge of the initial image is that the terminal judges the gray value of the pixel point by the change of the gray values of two adjacent pixels in the vertical or horizontal direction of the gradient magnitude (that is, the difference in gray values of two adjacent pixel points) to exclude some positions that do not meet the edge conditions but are determined as edge points during the terminal scanning.

[0159] S204. Obtain the minimum bounding rectangle of the initial image, and perform edge extraction according to the minimum bounding rectangle to determine the edge region;

[0160] The purpose of calculating the gradients of the initial image in the horizontal and vertical directions is to calculate the region containing the screen side features in the initial image, that is, to obtain the minimum bounding rectangle of the initial image, and perform edge extraction according to the minimum bounding rectangle to determine the edge region.

[0161] S205. Split the edge region according to the preset splitting density to obtain multiple analysis regions.

[0162] In an actual situation, the edge region appears as a rectangular frame on the initial image, that is, the minimum bounding rectangle. After pixel expansion of the minimum bounding rectangle, the terminal will split the minimum bounding rectangle according to a preset splitting density. Each of the split regions is an analysis region. Subsequently, the terminal will analyze each analysis region one by one. All the analysis regions split in this step are multiple analysis regions. The splitting density is a preset value. Setting the splitting density can obtain more points above the screen side by shrinking the analysis region, improving the accuracy of subsequent steps.

[0163] Specifically, the splitting density can be the number of pixels, distance, or other measurement criteria, which are not specifically limited here. The terminal splits the edge region determined in the previous step into smaller analysis regions according to the preset splitting density. All the analysis regions are collectively referred to as multiple analysis regions.

[0164] S206. Connect the midpoints of the two short sides of each analysis region respectively to obtain multiple reference lines;

[0165] S207. Obtain all the gray values of all the pixel points on multiple reference lines one by one;

[0166] Steps S206 to S207 in this embodiment are similar to steps S104 to S105 in the previous embodiment, and will not be elaborated here specifically.

[0167] S208. Traverse all the gray values in the positive direction and calculate the first gray value difference between adjacent two gray values among all the gray values to obtain a set of first gray value differences;

[0168] Specifically, there are two traversal directions: positive traversal and reverse traversal. Among them, positive traversal is to compare and calculate the gray values corresponding to all the pixel points on all the reference lines with the positive direction of the image as the traversal direction. The positive direction in the initial image is the direction from bottom to top of the image. Conversely, the negative direction in the initial image is the direction from top to bottom of the image, that is, the reverse traversal described in subsequent steps.

[0169] The gray value difference is the difference between the gray values corresponding to two adjacent pixel points. Generally, the first pixel point read is the first element of the difference, and the second pixel point read is the second element of the difference. The calculation method of the gray value difference is: gray value difference = gray value of the first element - gray value of the second element.

[0170] S209. Obtain the gray value differences in the first gray value difference set that are less than the first preset value, and record the pixel points corresponding to the gray value differences less than the first preset value as the first positive target point positions, obtaining the first positive target point position set. The first preset value is used to determine whether the pixel points corresponding to two gray values satisfy the changing state from dark to bright.

[0171] In actual situations, steps S106 and S107 are executed synchronously. That is, when the terminal calculates the gray value difference corresponding to a pair of adjacent pixel points, it will compare this gray value difference with the first preset value.

[0172] In this application, the first preset value is actually the threshold of the gray value difference (gray amplitude). The gray amplitude is the change amplitude of the gray values of two adjacent pixel points. Therefore, the gray amplitude between two adjacent pixel points and the gray value difference are the same data. The first preset value is used to determine whether the gray amplitude generated when the gray values of two adjacent pixel points change from dark to bright can identify the gray value change of this pixel point as an edge point position. This edge point position can be the edge of the actual screen side or the edge of an abnormal area inside the screen side.

[0173] Specifically, please refer to Figure 5 , Figure 5 is the schematic diagram of the initial image in this embodiment. Among them, the red block diagram is the actual state of one analysis area after the minimum circumscribed rectangle border is divided by a preset density under different inclination conditions. The red point positions and green point positions respectively correspond to the target point positions obtained by forward traversal and reverse traversal. In actual situations, the shooting scene is arranged according to standard rules, so that the initial image obtained by shooting in this shooting scene only contains the information of the screen side and the background information that is almost recognized as a solid color. The influence of noise on the image will be reduced to the minimum in the processed initial image, and the confirmation of the minimum circumscribed rectangle has been completed in the previous steps. The minimum circumscribed rectangle scans to obtain the edge information of the screen side included in the initial image, and in order to avoid information omission, the minimum circumscribed rectangle is expanded by a certain number of pixels, so that when the terminal detects the initial image, each recorded gray amplitude change occurs within the range of the minimum circumscribed rectangle. Within this range, if the gray amplitude change between two adjacent pixel points is less than the first preset value, it can indicate that the two pixel points are considered to be the edge points of a feature.

[0174] Therefore, after the terminal completes the comparison of all the gray value differences in the first gray value difference set with the first preset value, the terminal will record the point positions that meet the first preset value condition for the first gray value difference. These point positions are obtained by forward traversal, so they are the first positive target point position set.

[0175] S210. Obtain the gray - value differences in the first gray - value difference set that are greater than the second preset value, and record the pixel points corresponding to the gray - value differences greater than the second preset value as the second positive target point positions, obtaining the second positive target point - position set. The second preset value is used to determine whether the pixel points corresponding to two gray - values satisfy the change state from bright to dark.

[0176] Similar to the first preset value, the second preset value is used to determine whether the gray - value amplitude generated when the gray - values of two adjacent pixel points change from bright to dark can identify the change of the gray - value of this pixel point as an edge point position. This edge point position can be the edge of the actual screen side or the edge of the abnormal area inside the screen side.

[0177] After the terminal completes the comparison of all gray - value differences in the first gray - value difference set with the second preset value, the terminal will record the positions of the first gray - value differences that meet the second preset value condition. These positions are obtained through forward traversal, so they are the second positive target point - position set.

[0178] S211. Traverse all gray - values in reverse order, and calculate the second gray - value differences between two adjacent gray - values among all gray - values, obtaining the second gray - value difference set.

[0179] After completing the forward traversal, the terminal will traverse all gray - values in reverse order. The reverse traversal is to traverse the gray - values in the negative direction in the initial image as the direction from top to bottom of the image. The order of obtaining gray - values in reverse traversal is opposite to that in forward traversal. Therefore, according to the calculation method of gray - value difference: gray - value difference = gray - value of the first element−gray - value of the second element; it can be known that because the acquisition order of the first element and the second element is opposite to that in forward traversal, the obtained results will also be different.

[0180] S212. Obtain the gray - value differences in the second gray - value difference set that are less than the first preset value, and record the pixel points corresponding to the gray - value differences less than the first preset value as the first negative target point positions, obtaining the first negative target point - position set.

[0181] The judgment condition for the gray - value difference data obtained by reverse traversal is the same as that for the gray - value difference data obtained by forward traversal, and the order of pixel points obtained by reverse traversal is opposite to that by forward traversal. Therefore, the first negative target point - position set and the first positive target point - position set will not have overlapping points, and in the two sets, the actual number of positive target point positions and negative target point positions belonging to the same analysis area can be more than one.

[0182] All elements in the second gray - value difference set are screened by the first preset value, and the pixel points corresponding to all the recorded second gray - value differences are the first negative target point - position set.

[0183] S213. Obtain the gray value differences greater than the second preset value in the second gray value difference set, and record the pixel points corresponding to the gray value differences greater than the second preset value as the second negative target point positions, obtaining the second negative target point position set;

[0184] Filter all elements in the second gray value difference set through the second preset value, and the pixel points corresponding to all the recorded second gray value differences are the second negative target point position set.

[0185] In this embodiment, by performing two screenings on the gray value differences of adjacent gray values in the gray value set, four target point position sets, namely the first positive target point position set, the first negative target point position set, the second positive target point position set, and the second negative target point position set, are obtained. Among them, different two preset values are used to compare and screen the gray value difference data in the different gray value difference sets calculated by forward traversal and reverse traversal respectively, so as to screen out the suspicious points where the gray value amplitude suddenly changes from dark to bright or from bright to dark in the initial image. These suspicious points are the target point positions. In actual situations, the pixel point positions from dark to bright during forward traversal and from bright to dark during reverse traversal in the same analysis area can be the same point position.

[0186] S214. Obtain the overlapping point positions of the first positive target point position set and the second negative target point position set, obtaining the first overlapping point position set;

[0187] There is a corresponding relationship among the first positive target point position set, the first negative target point position set, the second positive target point position set, and the second negative target point position set obtained through the foregoing steps.

[0188] Specifically, the acquisition condition of the first positive target point position set is the elements of the first gray value difference set less than the first preset value. The first gray value difference set is obtained through forward traversal, and the screening rule of the first preset value is to obtain adjacent pixel points where the gray value amplitude changes from dark to bright;

[0189] Similarly, the acquisition condition of the second negative target point position set is the elements of the second gray value difference set greater than the second preset value. The second gray value difference set is obtained through reverse traversal, and the screening of the second preset value is to obtain adjacent pixel points where the gray value amplitude changes from bright to dark;

[0190] And according to Figure 5 it can be known that in the same analysis area, a pixel point from bright to dark and a pixel point from dark to bright can obtain a coincident pixel point under the screening conditions of two different preset values. This point position is the coincident point. Therefore, the terminal can screen out a set of coincident point positions from the data of the first positive target point position set and the second negative target point position set, that is, the first coincident point position.

[0191] S215. Obtain the overlapping points of the second set of positive target points and the first set of negative target points to obtain the second set of overlapping points;

[0192] Similar to the analysis of the first set of positive target points and the second set of negative target points mentioned above, because the pixel points from dark to bright during forward traversal and from bright to dark during reverse traversal in the same analysis area can be the same point, there are also overlapping points in the second set of positive target points and the first set of negative target points, and these overlapping points are the second set of overlapping points.

[0193] By determining the overlapping points, the points in the set of target points can be screened. The above steps have performed two traversals of two preset values in total, and the obtained target point data can correspond one by one. That is, in this embodiment, through two rules (the first preset value and the second preset value), the gray values corresponding to the pixel points on the baseline of the analysis area are traversed forward and backward respectively, and four sets of data that correspond one by one are obtained. By obtaining the overlapping data, the accuracy of the target points in the first overlapping point and the second overlapping point is improved.

[0194] S216. Determine the outermost edge points of the screen body from the first set of overlapping points and the second set of overlapping points according to the preset edge area.

[0195] Specifically, although the area obtained after pixel expansion of the minimum circumscribed rectangle determined in the above steps covers all the side information of the screen body, the side information of the screen body includes the actual contour of the original screen body side after expansion. Therefore, the actual outermost edge points are also included in the first set of overlapping points and the second set of overlapping points. According to the determination process of the above steps, it can be known that the first overlapping point and the second overlapping point finally obtained by forward traversal and reverse traversal of the initial image are used to obtain the upper and lower sides of the screen side contour respectively. Therefore, among the first overlapping point and the second overlapping point belonging to the same analysis area, the overlapping points with the same distance from the actual screen side parameters will be considered as the outermost edge points of the screen body side, that is, the outermost edge points of the screen body. After determining the upper and lower sides of the screen body, the left and right sides of the screen body side area are the same as the actual height of the screen body. Therefore, the two overlapping points whose distances between the overlapping points corresponding to the analysis area in the first set of overlapping points and the second set of overlapping points meet the actual height of the actual screen body will be considered as the upper edge and the lower edge of the analysis area respectively.

[0196] The terminal will analyze the overlapping points in all analysis areas one by one according to the above method, and then obtain the outermost edge points of the screen body in all analysis areas.

[0197] S217. Determine the coordinate values of the pixel points corresponding to the outermost edge points of the screen body on the initial image, and establish a least squares method model according to the coordinate values;

[0198] The terminal obtains the coordinates of the outermost edge points of the screen body on the initial image. Among them, since the height of the side of the screen body is determined, and the screen body may have an angle during actual shooting, when the coordinates of the outermost edge points are , where is the independent variable, is the dependent variable.

[0199] Establish a least squares method model based on the above coordinates. Specifically, by minimizing the predicted value and the observed value to find the optimal and , where is the slope, is the intercept.

[0200] S218. Calculate the sum of squared errors between the minimized predicted value and the observed value of the least squares method model, and determine the slope and intercept of the fitted target line;

[0201] Specifically, the formula for the sum of squared errors is:

[0202]

[0203] From the formula for the sum of squared errors, it can be seen that to determine the edge of the actual detection area, the purpose of the above formula is to find m and b that minimize SSE. Therefore, after determining the formula, the partial derivatives of m and b need to be calculated separately and set to 0. The derivation process is as follows:

[0204]

[0205]

[0206] After organizing the above formula, the expressions of m and b are obtained, which are respectively:

[0207]

[0208]

[0209] S219. Determine the actual detection area on the initial image according to the slope and intercept.

[0210] After obtaining the above expressions, substitute the coordinate values of the outermost edge points of the screen body actually obtained from the initial image to find m and b in the linear fitting target equation . After obtaining the result of linear fitting, the edge box obtained after closing the line fitted on the initial image according to the parameters of the actual screen body side is the actual outermost edge of the screen body, that is, the area where abnormalities actually need to be detected.

[0211] S220. Obtain the overlapping points of the first overlapping point set and the second overlapping point set within the actual detection area to obtain the edge points of the abnormal area.

[0212] In the foregoing steps, the terminal generated a new edge frame of the screen side in the initial image according to the traversed grayscale value difference. This edge frame of the screen side fits better with the side edge to be detected in the actual initial image than the minimum bounding rectangle in the foregoing steps, and the terminal can exclude the interference of the overlapping points outside the screen edge area and the screen overlapping area on the subsequent abnormal detection according to the actual detection area.

[0213] At this time, the terminal will determine the overlapping points belonging to the actual detection area among all the overlapping points in the first overlapping point set and the second overlapping point set. The actual coordinates of these overlapping points in the initial image must fall within the actual detection area.

[0214] S221. Analyze the height differences of adjacent overlapping points one by one from left to right according to the position relationship of the edge points of the abnormal area on the initial image;

[0215] Since all the coordinate points in the edge points of the abnormal area must be within the actual detection area on the initial image, and the point set of the edge points of the abnormal area is the same as the first overlapping point set and the second overlapping point set, and the actual points are distinguished for different overlapping points through the analysis area, that is, the overlapping points must be on the baseline. Therefore, after determining the edge points of the abnormal area, the terminal will traverse all the overlapping points in the edge points of the abnormal area from left to right according to the analysis area. Among them, the adjacent points will be compared for height difference, and whether the overlapping points are at the same level will be determined according to the height differences of these overlapping points.

[0216] S222. When the height difference is within the error range of the same level, determine that the adjacent overlapping points used to calculate the height difference belong to the same level;

[0217] Essentially, the abnormal area is also an area. Therefore, the fitting results of the upper and lower sides of the abnormal area can also be obtained in the foregoing manner, and the defective area can be obtained by closing the fitting results. Therefore, at this time, the terminal needs to analyze the height differences of the overlapping points of two adjacent analysis areas to distinguish the points on the upper side and the lower side of the abnormal area. The points belonging to the upper side and the points belonging to the lower side do not belong to the same level.

[0218] S223. Determine all the levels in the actual detection area that contain the edge points of the abnormal area;

[0219] Generally, an abnormal area includes an upper edge and a lower edge. Multiple levels mean that there is more than one abnormal area within the actual detection area. According to the description of the foregoing steps, the coincident points among the edge points of the abnormal area are distinguished by analysis area. Therefore, the terminal can determine the levels of the edge points of the abnormal area based on the actual coordinate positions of the coincident points in each analysis area.

[0220] However, in actual situations, the shooting of the screen side may cause the screen side to be tilted in the image. Therefore, when determining the level of the coincident points on the screen side, the terminal will determine a preset value according to the gradient direction calculated in the foregoing steps (generally, the set threshold should be very large, but not exceed the height of the screen side). When the height difference (y-axis difference) between two adjacent coincident points meets the preset value, it is determined that these two adjacent coincident points belong to the same level.

[0221] S224. Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area;

[0222] After determining all the coincident points of the same level, connect all the coincident points recognized as the same level. The specific connection process is to fit the upper and lower sides of the actual detection area through the foregoing fitting steps to determine the actual position of the edge of the abnormal area.

[0223] The purpose of fitting is to determine the connection relationship between two adjacent pixel points. The difference between the edge of the abnormal area and the edge of the actual detection area is that the actual detection area must be a continuous straight line, but the edge of the abnormal area can be an irregular shape. Therefore, the actual fitting process needs to fit adjacent points one by one.

[0224] After completing the fitting of all the coincident points on all levels, the edge of the abnormal area is obtained.

[0225] S225. Close the edge of the abnormal area to obtain the abnormal area.

[0226] The edge of the abnormal area in the foregoing steps is the edge of the abnormal area based on levels. Therefore, a step of closing the edge of the abnormal area is required to determine the complete abnormal area.

[0227] It should be noted that after determining the abnormal area on the screen side in the initial image, the terminal will input the image carrying the abnormal area information into the classification network to analyze the side abnormality, further assisting the operator to judge the cause of the abnormality. The specific analysis process is as follows:

[0228] Input the target image with abnormal area data into the classification network so that the classification network analyzes the target image and outputs the cause of the abnormality.

[0229] In practical applications, to determine the cause of an anomaly through a classification network, it is first necessary to collect and preprocess image data, including operations such as scaling and normalizing the images to meet the input requirements of the network. After determination, a pre-trained deep learning model or a custom network structure is used to extract and analyze the features of the images. Through training, the model can learn the feature representations of normal images and, based on this, identify abnormal regions that are significantly different from the normal features.

[0230] The classification network used in this application is a learning network that has completed basic learning for classifying anomalies on the side of the screen, and the classification network is directly arranged on the terminal. Therefore, after the terminal marks the abnormal region on the side of the screen, when the image with the edge of the abnormal region is input into the classification network model, the classification network model will determine the abnormal type corresponding to the abnormal region of the side screen image through the following analysis process:

[0231] First, the terminal obtains all abnormal regions in the side screen image according to the foregoing steps, cuts these abnormal regions from the image according to the edges of the abnormal regions, and inputs the cut defective regions into the classification network for classification to obtain the classification results of the classification network outputting all classifications of anomalies based on the input abnormal region images. The classifications include but are not limited to: dirt, scratches, wrinkles, burns.

[0232] The terminal will generate a data list (including all feature parameters) according to the feedback of the classification network, replace the classification results into the data list, and finally the terminal outputs the data list data.

[0233] This embodiment is a complete process from the analysis of the initial image to the output of anomalies on the side of the screen, which specifically describes the method for determining the actual region on the side of the screen and the method for determining the abnormal region on the side of the screen. After the determination of the abnormal region is completed, the terminal will also input these images into the classification model and output the possible causes of the anomalies corresponding to the abnormal information contained in the images, thereby realizing a complete automatic detection closed-loop for anomalies on the side of the screen and improving the efficiency and accuracy of detecting anomalies on the side of the screen.

[0234] The above has described in detail the method for detecting anomalies on the side of the screen body in the embodiments of the present application. Next, the anomaly detection system and device for the side of the screen body will be described in detail.

[0235] Please refer to Figure 3 , an embodiment of the anomaly detection system for the side of the screen body provided by the embodiments of the present application includes:

[0236] A photographing unit 301, configured to photograph an initial image of the side of the display screen;

[0237] The first determination unit 302 is configured to determine the edge region of the initial image;

[0238] The splitting unit 303 is configured to split the edge region according to a preset splitting density to obtain a plurality of analysis regions;

[0239] The connection unit 304 is configured to connect the midpoints of the two short sides of each of the analysis regions respectively to obtain a plurality of reference lines;

[0240] The acquisition unit 305 is configured to sequentially acquire all the gray values of all the pixel points on the plurality of reference lines;

[0241] The first traversal unit 306 is configured to traverse all the gray values according to a preset rule to determine the outermost edge points of the screen body;

[0242] The first fitting unit 307 is configured to fit the outermost edge points of the screen body to obtain an actual detection region;

[0243] The second traversal unit 308 is configured to traverse all the gray values within the actual detection region according to the preset rule to determine the edge points of the abnormal region;

[0244] The second fitting unit 309 is configured to fit the edge points of the abnormal region to obtain an abnormal region.

[0245] Optionally, the first traversal unit 306 is specifically configured to:

[0246] Traverse all the gray values in the forward direction, and calculate the first gray value difference between two adjacent gray values among all the gray values to obtain a first set of gray value differences;

[0247] Obtain the gray value differences in the first set of gray value differences that are less than a first preset value, and record the pixel points corresponding to the gray value differences that are less than the first preset value as the first forward target points to obtain a first set of first forward target points. The first preset value is used to determine whether the pixel points corresponding to two gray values satisfy the state of change from dark to bright;

[0248] Obtain the gray value differences in the first set of gray value differences that are greater than a second preset value, and record the pixel points corresponding to the gray value differences that are greater than the second preset value as the second forward target points to obtain a second set of second forward target points. The second preset value is used to determine whether the pixel points corresponding to two gray values satisfy the state of change from bright to dark;

[0249] Traverse all the gray values in the reverse direction, and calculate the second gray value difference between two adjacent gray values among all the gray values to obtain a second set of gray value differences;

[0250] Obtain the gray value differences in the second gray value difference set that are less than the first preset value, and record the pixel points corresponding to the gray value differences less than the first preset value as the first negative target points to obtain the first negative target point set;

[0251] Obtain the gray value differences in the second gray value difference set that are greater than the second preset value, and record the pixel points corresponding to the gray value differences greater than the second preset value as the second negative target points to obtain the second negative target point set;

[0252] Obtain the overlapping points of the first positive target point set and the second negative target point set to obtain the first overlapping point set;

[0253] Obtain the overlapping points of the second positive target point set and the first negative target point set to obtain the second overlapping point set;

[0254] Determine the outermost edge points of the screen body from the first overlapping point set and the second overlapping point set according to the preset edge area.

[0255] Optionally, the second traversal unit 308 is specifically configured to:

[0256] Obtain the overlapping points of the first overlapping point set and the second overlapping point set within the actual detection area to obtain the edge points of the abnormal area.

[0257] Optionally, the first fitting unit 307 is specifically configured to:

[0258] Determine the coordinate values of the pixel points corresponding to the outermost edge points of the screen body on the initial image, and establish a least squares model according to the coordinate values;

[0259] Calculate the sum of the squared errors between the minimized predicted values and the observed values of the least squares model, and determine the slope and intercept of the fitting target line;

[0260] Determine the actual detection area on the initial image according to the slope and intercept.

[0261] Optionally, the splitting unit 303 is specifically configured to:

[0262] Obtain the minimum bounding rectangle of the initial image, and perform edge extraction according to the minimum bounding rectangle to determine the edge area;

[0263] Split the edge area according to the preset splitting density to obtain a plurality of analysis areas.

[0264] Optionally, the splitting unit 303 is specifically further configured to:

[0265] Calculate the gradients of the initial image in the horizontal and vertical directions using the Sobel operator to obtain the gradient magnitude and gradient direction;

[0266] Scan the initial image according to the gradient magnitude and the gradient direction, and obtain and connect pixel points that satisfy the gradient magnitude on the initial image one by one to obtain an edge region.

[0267] Optionally, the second fitting unit 309 is specifically configured to:

[0268] Analyze the height differences of adjacent overlapping points one by one from left to right according to the positional relationship of the edge points of the abnormal region on the initial image;

[0269] When the height difference is within the error range of the same level, determine that the adjacent overlapping points used to calculate the height difference belong to the same level;

[0270] Determine all levels in the actual detection region that contain the edge points of the abnormal region;

[0271] Connect the edge points of the abnormal region on each level one by one to obtain the edge of the abnormal region;

[0272] Close the edge of the abnormal region to obtain the abnormal region.

[0273] In this embodiment, the functions of each unit correspond to the steps in the foregoing Figure 1 and Figure 2a 、 Figure 2b 、 Figure 2c illustrated embodiments, and will not be elaborated here.

[0274] Please refer to Figure 4 , another embodiment of the abnormal detection device on the side of the screen body provided by the embodiment of the present application includes:

[0275] A processor 401, a memory 402, an input / output unit 403, and a bus 404;

[0276] The processor 401 is connected to the memory 402, the input / output unit 403, and the bus 404;

[0277] The processor 401 specifically executes the operations corresponding to the steps in Figure 1 and Figure 2a 、 Figure 2b 、 Figure 2c , and will not be elaborated here specifically.

[0278] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program runs on a computer, the computer is enabled to execute any of the above methods.

[0279] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0280] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0281] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0282] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0283] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs and other media that can store program codes.

Claims

1. A method for detecting abnormalities on the side of a screen, characterized in that: Said: Take an initial image of the side of the display; Determining an edge region of the initial image; Splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions; Connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; Obtaining all grayscale values ​​of all pixels on the multiple reference lines one by one; Traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; Fitting the outermost edge points of the screen to obtain the actual detection area; Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to the grayscale value differences greater than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; Determine the outermost edge point of the screen from the first set of coincident points and the second set of coincident points according to a preset edge area; Fit the edge points of the abnormal region to obtain the abnormal region.

2. The anomaly detection method according to claim 1, characterized in that: Traversing all grayscale values ​​in the actual detection area according to the preset rule to determine the edge points of the abnormal area includes: Obtain the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area to obtain edge points of the abnormal area.

3. The method according to claim 1, characterized in that The step of fitting the outermost edge point of the screen to obtain the actual detection area includes: Determine the coordinate values ​​of the pixel points corresponding to the outermost edge points of the screen on the initial image, and establish a least squares model according to the coordinate values; Calculate the sum of squares of the minimized predicted value and the observed value of the least squares model to determine the slope and intercept of the fitted target straight line; An actual detection area is determined on the initial image according to the slope and the intercept.

4. The method according to claim 1, characterized in that: The edge region is split according to a preset splitting density to obtain a plurality of analysis regions, including: Obtaining a minimum bounding rectangle of the initial image, and performing edge extraction based on the minimum bounding rectangle to determine an edge area; The edge region is split according to a preset splitting density to obtain a plurality of analysis regions.

5. The method according to claim 4, characterized in that The step of obtaining a minimum bounding rectangle of the initial image and performing edge extraction according to the minimum bounding rectangle to determine an edge area includes: Using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction, and obtain the gradient amplitude and the gradient direction; The initial image is scanned according to the gradient amplitude and the gradient direction, and pixel points satisfying the gradient amplitude on the initial image are acquired one by one and connected to obtain an edge region.

6. The method according to any one of claims 1 to 5, characterized in that The step of fitting the edge points of the abnormal region to obtain the abnormal region includes: Analyze the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; When the height difference is within the error range of the same level, it is determined that the adjacent coincident points used to calculate the height difference belong to the same level; Determine all levels in the actual detection area that contain edge points of the abnormal area; Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; The edge of the abnormal region is closed to obtain the abnormal region.

7. A screen side abnormality detection system, characterized in that: The system comprises: A shooting unit, used for shooting an initial image of a side surface of the display screen; A first determining unit, configured to determine an edge region of the initial image; A splitting unit, used for splitting the edge area according to a preset splitting density to obtain a plurality of analysis areas; A connecting unit, used for connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; An acquisition unit, used for acquiring all grayscale values ​​of all pixels on the plurality of reference lines one by one; A first traversal unit, used for traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; A first fitting unit, used for fitting the outermost edge points of the screen to obtain an actual detection area; A second traversal unit, used to traverse all gray values ​​in the actual detection area according to the preset rule to determine the edge points of the abnormal area; The second fitting unit is used to fit the edge points of the abnormal area to obtain the abnormal area; The first traversal unit is specifically used for: Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to the grayscale value differences greater than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; The outermost edge point of the screen is determined from the first set of coincident points and the second set of coincident points according to a preset edge area.

8. A device for detecting abnormalities on the side of a screen, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 6.

Images