Screen body side face anomaly detection method, system and device

Abstract

The invention discloses a screen body side anomaly detection method, system and device, which are used for improving the efficiency of screen body side anomaly detection. The method comprises the steps of shooting an initial image of the side surface of a display screen; determining an edge region of the initial image; splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions; connecting the midpoints of the two short edges of each analysis area to obtain a plurality of datum lines; obtaining all gray values of all pixel points on the plurality of datum lines one by one; traversing all the gray values according to a preset rule, and determining the outermost edge point of the screen body; fitting the outermost edge point of the screen body to obtain an actual detection area; traversing all gray values in the actual detection area according to the preset rule, and determining edge points of an abnormal area; and fitting the edge points of the abnormal region to obtain the abnormal region.

Description

Technical Field

[0001] The present application relates to the field of screen detection, and in particular to a method, system and device for detecting abnormalities on the side of a screen. Background Art

[0002] With the development of display technology, display screens are widely used in various electronic devices. However, in actual use, display screens may have problems on the side of the screen due to external forces, aging, etc., such as cracks, deformation, or display abnormalities. If these problems are not detected and handled in time, they may affect the overall performance of the device or even cause serious damage.

[0003] In the prior art, display screen detection methods are mostly concentrated on the central area of ​​the screen, and detection of the sides of the screen is relatively rare, resulting in low efficiency in abnormality detection on the sides of the screen. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a method, system and device for detecting abnormalities on the side of a screen, which are used to improve the efficiency of detecting abnormalities on the side of a screen.

[0005] The technical solution provided in this application is described below: The first aspect of the present application provides a method for detecting an abnormality on the side of a screen, comprising: Take an initial image of the side of the display; Determining an edge region of the initial image; Splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions; Connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; Obtaining all grayscale values ​​of all pixels on the multiple reference lines one by one; Traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; Fitting the outermost edge points of the screen to obtain the actual detection area; Traversing all grayscale values ​​in the actual detection area according to the preset rule to determine edge points of the abnormal area; The edge points of the abnormal region are fitted to obtain the abnormal region.

[0006] Optionally, traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen includes: Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to grayscale value differences less than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; The outermost edge point of the screen is determined from the first set of coincident points and the second set of coincident points according to a preset edge area.

[0007] Optionally, traversing all grayscale values ​​in the actual detection area according to the preset rule to determine edge points of the abnormal area includes: Obtain the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area to obtain edge points of the abnormal area.

[0008] Optionally, fitting the outermost edge point of the screen to obtain the actual detection area includes: Determine the coordinate value of the pixel point corresponding to the outermost edge point of the screen on the initial image, and establish a least squares model according to the coordinate value; Calculate the sum of squares of the minimized predicted value and the observed value of the least squares model to determine the slope and intercept of the fitted target straight line; An actual detection area is determined on the initial image according to the slope and the intercept.

[0009] Optionally, the edge region is split according to a preset splitting density to obtain a plurality of analysis regions, including: Obtaining a minimum bounding rectangle of the initial image, and performing edge extraction based on the minimum bounding rectangle to determine an edge area; The edge region is split according to a preset splitting density to obtain a plurality of analysis regions.

[0010] Optionally, the acquiring of a minimum bounding rectangle of the initial image and performing edge extraction to determine an edge region according to the minimum bounding rectangle includes: Using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction, and obtain the gradient amplitude and the gradient direction; The initial image is scanned according to the gradient amplitude and the gradient direction, and pixel points satisfying the gradient amplitude on the initial image are acquired one by one and connected to obtain an edge region.

[0011] Optionally, fitting the edge points of the abnormal region to obtain the abnormal region includes: Analyze the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; When the height difference is within the error range of the same level, it is determined that the adjacent coincident points used to calculate the height difference belong to the same level; Determine all levels in the actual detection area that contain edge points of the abnormal area; Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; The edge of the abnormal region is closed to obtain the abnormal region.

[0012] The second aspect of the present application provides a screen side abnormality detection system, comprising: A shooting unit, used for shooting an initial image of a side surface of the display screen; A first determining unit, configured to determine an edge region of the initial image; A splitting unit, used for splitting the edge area according to a preset splitting density to obtain a plurality of analysis areas; A connecting unit, used for connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; An acquisition unit, used for acquiring all grayscale values ​​of all pixels on the plurality of reference lines one by one; A first traversal unit, used for traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; A first fitting unit, used for fitting the outermost edge points of the screen to obtain an actual detection area; A second traversal unit, used to traverse all gray values ​​in the actual detection area according to the preset rule to determine the edge points of the abnormal area; The second fitting unit is used to fit the edge points of the abnormal region to obtain the abnormal region.

[0013] Optionally, the first traversal unit is specifically used for: Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to grayscale value differences less than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; The outermost edge point of the screen is determined from the first set of coincident points and the second set of coincident points according to a preset edge area.

[0014] Optionally, the second traversal unit is specifically used for: Obtain the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area to obtain edge points of the abnormal area.

[0015] Optionally, the first fitting unit is specifically used for: Determine the coordinate value of the pixel point corresponding to the outermost edge point of the screen on the initial image, and establish a least squares model according to the coordinate value; Calculate the sum of squares of the minimized predicted value and the observed value of the least squares model to determine the slope and intercept of the fitted target straight line; An actual detection area is determined on the initial image according to the slope and the intercept.

[0016] Optionally, the splitting unit is specifically used for: Obtaining a minimum bounding rectangle of the initial image, and performing edge extraction based on the minimum bounding rectangle to determine an edge area; The edge region is split according to a preset splitting density to obtain a plurality of analysis regions.

[0017] Optionally, the splitting unit is further used for: Using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction, and obtain the gradient amplitude and the gradient direction; The initial image is scanned according to the gradient amplitude and the gradient direction, and pixel points satisfying the gradient amplitude on the initial image are acquired one by one and connected to obtain an edge region.

[0018] Optionally, the second fitting unit is specifically used for: Analyze the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; When the height difference is within the error range of the same level, it is determined that the adjacent coincident points used to calculate the height difference belong to the same level; Determine all levels in the actual detection area that contain edge points of the abnormal area; Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; The edge of the abnormal region is closed to obtain the abnormal region.

[0019] A third aspect of the present application provides a device for detecting an abnormality on the side of a screen, the device comprising: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0021] It can be seen from the above technical solutions that this application has the following advantages: Take an initial image of the side of the display; Determining an edge region of the initial image; Splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions; Connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; Obtaining all grayscale values ​​of all pixels on the multiple reference lines one by one; Traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; Fitting the outermost edge points of the screen to obtain the actual detection area; Traversing all grayscale values ​​in the actual detection area according to the preset rule to determine edge points of the abnormal area; The edge points of the abnormal region are fitted to obtain the abnormal region.

[0022] An initial image of the side of the screen is obtained, and the edge area of ​​the side of the screen in the initial image is segmented to determine a baseline for refining the edge of the side of the screen in the initial image. The actual detection area is determined by traversing the grayscale values ​​of the pixels on each baseline, and the grayscale values ​​of the pixels on the baseline are analyzed again according to the actual detection area to obtain the edge points of the abnormal area. The abnormal points are connected by fitting to obtain the abnormal area determined by all the abnormal points in the actual side area. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a schematic diagram of an embodiment of a method for detecting abnormalities on the side of a screen in the present application; Figure 2a This is another embodiment flow chart of the first stage of the abnormality detection method for the side of the screen in the present application; Figure 2b This is another embodiment flow chart of the second stage of the abnormality detection method for the side of the screen in the present application; Figure 2cThis is another embodiment flow chart of the third stage of the abnormality detection method for the side of the screen in the present application; Figure 3 This is a schematic diagram of the structure of an embodiment of an abnormality detection system for the side of a screen in the present application; Figure 4 This is a schematic diagram of the structure of an embodiment of an abnormality detection device for the side of a screen body in the present application; Figure 5 This is a schematic diagram of the initial image in the abnormality detection method on the side of the screen in this application. DETAILED DESCRIPTION

[0025] It should be noted that the method for detecting anomalies on the side of a screen provided in this application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses a terminal as an example for explanation.

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] See also Figure 1 The present application first provides an embodiment of a method for detecting an abnormality on the side of a screen, the embodiment comprising: S101, capturing an initial image of the side of the display screen; The initial image is an image of the sides of the screen captured by an industrial camera after the actual screen production is completed and the screen body is assembled around it.

[0028] In the actual shooting environment, the lens parameters of the industrial camera used for shooting are fixed, so that the aperture focal length of the industrial camera is maintained in a state that meets the working environment of the shooting system. The parameters that need to be adjusted are the position relationship between the platform and the camera, and the relative position of the point light source and the camera lens. Generally, the point light source is set on one side of the industrial camera lens. The above parameters are recorded in the terminal in actual situations. In actual operation, the camera can be directly controlled through the terminal, so that the camera can automatically obtain the corresponding parameters for setting after it is turned on. After determining the parameters of the industrial camera, the screen is adsorbed by the adsorption device, and the surrounding images of the side of the screen are collected by the rotation of the mechanical arm. These images are the initial images of the screen.

[0029] S102, determining the edge area of ​​the initial image; The terminal determines the edge area of ​​the screen side data in the initial image by calculating the gradient amplitude and gradient direction of each point in the initial image, and performs maximum value suppression on the edge area.

[0030] After the terminal completes non-maximum suppression, it will track the pixels that satisfy the grayscale amplitude change along the gradient direction. If the weak edge pixel is connected to the strong edge, it will be retained as the true edge, otherwise it will be discarded, and finally the complete edge area will be output.

[0031] A strong edge is an edge with a noticeable brightness change in an image, usually corresponding to the outline of an object or the boundary between an object and the background. A strong edge appears as a high gradient value in a gradient magnitude image, i.e., a rapid brightness change. In the output of an edge detection algorithm, strong edges usually appear as bright or dark lines because they stand out in the image. Strong edges are very important for image understanding and object recognition because they usually mark the boundaries of objects.

[0032] Weak edges are areas in an image where the brightness changes less significantly, which may correspond to subtle changes or textures on the surface of an object. Weak edges appear as low gradient values ​​in the gradient magnitude image, i.e., the brightness changes slowly. Weak edges may not be very obvious in the output of edge detection algorithms because they have low gradient values. Weak edges may contain useful information in some cases, such as texture analysis or pattern recognition, but they are usually treated as noise in most edge detection tasks.

[0033] The goal of edge detection algorithms is to identify and highlight strong edges in an image while suppressing weak edges and noise as much as possible. This is because strong edges usually contain important information about the image structure and object shape.

[0034] In practical applications, edge detection algorithms (such as Sobel, Canny, Laplacian, etc.) usually use some thresholds to distinguish strong edges from weak edges. Edges exceeding the threshold are considered strong edges, while edges below the threshold are considered weak edges or noise. By adjusting the threshold, the algorithm's sensitivity to edges can be controlled to meet different application requirements.

[0035] After determining the real area, the terminal generates a minimum enclosing rectangle for the real area, and expands the minimum enclosing rectangle by a certain number of pixels according to the preset value to obtain a rectangular frame containing all the screen side information. The rectangular frame is the edge area. After determining the edge area, the terminal will split the edge area according to the preset splitting density, where each split area is an analysis area, and all analysis areas are integrated into multiple analysis areas.

[0036] It should be noted that before determining the edge area of ​​the initial image, the initial image should be preprocessed. The specific preprocessing method is as follows: The purpose of preprocessing is to make the information carried by the initial image clearer, that is, to perform noise reduction on the initial image to filter out or even eliminate interference information.

[0037] The process of preprocessing the initial image includes at least two steps: mean filtering and Gaussian filtering. The mean filtering method is to filter the initial image by using a 3*3 mean filtering window to obtain a grayscale image corresponding to the initial image, and then use a Gaussian function as a filter to perform weighted averaging on the signal on the image to smooth the signal and reduce noise.

[0038] After the terminal obtains the grayscale image, it will perform Gaussian filtering on the grayscale image. Gaussian filtering is actually filtering the initial image after mean filtering by inputting a Gaussian convolution kernel. The Gaussian convolution kernel used in this embodiment is as follows:

[0039] Among them, G(x,y) is the grayscale value of the grayscale image at the coordinate (x,y), that is, x and y are the grayscale values ​​of the pixels corresponding to the coordinate parameters of the image. It is the standard deviation of the Gaussian distribution, which determines the smoothness of the filter. The filtered image will remove most of the noise and retain the main image features.

[0040] S103, splitting the edge region according to a preset splitting density to obtain multiple analysis regions; The terminal determines the edge area of ​​the screen side data in the initial image by calculating the gradient amplitude and gradient direction of each point in the initial image, and performs maximum value suppression on the edge area.

[0041] After the terminal completes non-maximum suppression, it will track the pixels that satisfy the grayscale amplitude change along the gradient direction. If the weak edge pixel is connected to the strong edge, it will be retained as the true edge, otherwise it will be discarded, and finally the complete edge area will be output.

[0042] A strong edge is an edge with a noticeable brightness change in an image, usually corresponding to the outline of an object or the boundary between an object and the background. A strong edge appears as a high gradient value in a gradient magnitude image, i.e., a rapid brightness change. In the output of an edge detection algorithm, strong edges usually appear as bright or dark lines because they stand out in the image. Strong edges are very important for image understanding and object recognition because they usually mark the boundaries of objects.

[0043] Weak edges are areas in an image where the brightness changes less significantly, which may correspond to subtle changes or textures on the surface of an object. Weak edges appear as low gradient values ​​in the gradient magnitude image, i.e., the brightness changes slowly. Weak edges may not be very obvious in the output of edge detection algorithms because they have low gradient values. Weak edges may contain useful information in some cases, such as texture analysis or pattern recognition, but they are usually treated as noise in most edge detection tasks.

[0044] The goal of edge detection algorithms is to identify and highlight strong edges in an image while suppressing weak edges and noise as much as possible. This is because strong edges usually contain important information about the image structure and object shape.

[0045] In practical applications, edge detection algorithms (such as Sobel, Canny, Laplacian, etc.) usually use some thresholds to distinguish strong edges from weak edges. Edges exceeding the threshold are considered strong edges, while edges below the threshold are considered weak edges or noise. By adjusting the threshold, the algorithm's sensitivity to edges can be controlled to meet different application requirements.

[0046] After determining the real area, the terminal generates a minimum enclosing rectangle for the real area, and expands the minimum enclosing rectangle by a certain number of pixels according to the preset value to obtain a rectangular frame containing all the screen side information. The rectangular frame is the edge area. After determining the edge area, the terminal will split the edge area according to the preset splitting density, and each split area is an analysis area.

[0047] S104, respectively connecting the midpoints of the two short sides of each analysis area to obtain a plurality of baselines; After determining the analysis area, the terminal will determine the midpoints of the short sides of these analysis areas one by one. The length of the short side of the analysis area can be calculated by the actual length of the long side of the minimum circumscribed rectangle on the initial image and the split density. Because the minimum circumscribed rectangle is essentially a rectangle, all analysis areas are also rectangles, and each analysis area is of the same size. Therefore, the baseline obtained by connecting the midpoints of the short sides is evenly distributed on the initial image. The baseline is used to represent the actual pixel distribution of the corresponding analysis area, that is, pixel sampling is uniformly performed at a fixed density on the initial image, and the baseline is obtained by connecting the midpoints of the upper and lower short sides of the analysis area. The number of baselines is consistent with the number of analysis areas, and one baseline represents the pixel distribution of one analysis area.

[0048] S105, obtaining all grayscale values ​​of all pixels on multiple baselines one by one; In the aforementioned steps, the terminal will traverse and scan the data carried on the image more than once to determine the noise reduction of the image. In actual situations, the border (i.e., the minimum enclosing rectangle mentioned above) obtained by directly scanning the image obtained by shooting is a rough estimated border that must cover all the screen side data.

[0049] S106, traversing all grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; Specifically, when traversing the grayscale value, the terminal will judge whether the pixel is an edge point based on the grayscale value difference. When the grayscale value difference is greater than the preset value, it means that the point is the outermost edge point of the screen. In actual situations, the grayscale value traversal needs to go through at least one forward traversal and one reverse traversal to improve the reliability of the outermost edge point of the screen.

[0050] S107, fitting the outermost edge point of the screen to obtain the actual detection area; After the terminal determines the outermost edge points of the screen in all analysis areas, the terminal will establish a least squares model based on these outermost edge points of the screen to fit the actual positions of the upper and lower sides of the screen in the initial image based on these outermost edge points of the screen, and close the actual positions of the upper and lower sides to obtain the actual detection area.

[0051] S108, traversing all gray values ​​in the actual detection area according to preset rules to determine edge points of the abnormal area; In the above steps, the terminal generates a new screen side edge frame in the initial image, namely, the actual detection area. The actual detection area is more closely aligned with the minimum circumscribed rectangle in the above steps and the side edge to be detected in the actual initial image, and the terminal can eliminate the interference of overlapping points outside the screen edge area and the screen overlap area on subsequent abnormal detection based on the actual detection area.

[0052] S109: Fit the edge points of the abnormal region to obtain the abnormal region.

[0053] The fitting process of the abnormal area is similar to that of the actual detection area, that is, the least squares model is established through the edge points of the abnormal area to fit the actual abnormal area.

[0054] See also Figure 2a , Figure 2b and Figure 2c The present application provides another embodiment of a method for detecting an abnormality on a side of a screen, the embodiment comprising: S201, capturing an initial image of the side of the display screen; Step S201 in this embodiment is similar to step S101 in the aforementioned embodiment, and will not be described in detail here.

[0055] S202, using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction to obtain the gradient amplitude and the gradient direction; In actual situations, the position of the side of the screen on the initial image is not an absolutely upright image, that is, the side of the screen may have an inclination on the initial image. Therefore, after determining the calculation method of the Gaussian convolution kernel, it is also necessary to calculate the actual position of the side of the screen on the initial image, specifically, to calculate the gradient Gx of the screen side image in the horizontal direction and the gradient Gy of the screen side image in the vertical direction in the initial image. Gx is the gradient component of the image in the horizontal direction (x direction), which represents the brightness change rate of the image in the horizontal direction; Gy is the gradient component of the image in the vertical direction (y direction), which represents the brightness change rate of the image in the vertical direction. For each pixel point (x, y) in the image, multiply its neighborhood pixel value by the weight in the corresponding Sobel operator, and then sum them to obtain the horizontal gradient Gx and vertical gradient Gy of the point.

[0056] Among them, Gx and Gy can be calculated by the Sobel operator, specifically by the following formula:

[0057]

[0058] Among them, G is the calculation formula of gradient amplitude, is the formula for the gradient direction.

[0059] S203, scanning the initial image according to the gradient amplitude and the gradient direction, acquiring and connecting pixel points satisfying the gradient amplitude on the initial image one by one, and obtaining an edge region.

[0060] After determining the direction and gradient of the image content on the side of the screen in the initial image according to the Sobel operator, the terminal scans the gradient amplitude of the image according to the gradient direction, and compares the gradient amplitude of the currently scanned pixel and its two neighboring pixels to see whether they meet the gradient amplitude G calculated according to the above formula. If not, the gradient amplitude is recorded as 0, that is, the grayscale gradient amplitude in the horizontal or vertical direction is suppressed to the maximum value.

[0061] The terminal will start tracking along the gradient direction from the strong edge determined by non-maximum suppression. The strong edge is the actual edge of the side of the screen determined by the terminal according to the gradient amplitude when performing the initial image scanning. However, for grayscale images, there may be a situation where the complete strong edge cannot be obtained due to the light and shooting direction of the shooting environment. Therefore, the terminal needs to determine whether the subsequent edge that is not determined as a strong edge is connected to the current strong edge. If it is connected, the subsequent edge is retained as a real edge, otherwise it is discarded and the complete edge is finally output. After determining the complete edge, the terminal will expand the upper and lower edges of the complete edge by a certain number of pixels according to the preset value to ensure that the actual screen side information can be fully contained in the minimum circumscribed rectangle. At this time, the complete edges obtained are connected to each other to determine the edge area of ​​the initial image. This edge area is the edge area in the initial image that contains all the screen side information.

[0062] Specifically, the terminal determines the strong edge of the initial image in the following process: the terminal determines the gray value of the pixel through the change of the gray value of two adjacent pixels in the vertical or horizontal direction by the gradient amplitude (i.e., the gray value difference between two adjacent pixels), so as to exclude some points that do not meet the edge conditions but are determined to be edge points when the terminal is scanning.

[0063] S204, obtaining the minimum bounding rectangle of the initial image, and performing edge extraction according to the minimum bounding rectangle to determine the edge area; The purpose of calculating the gradient of the initial image in the horizontal and vertical directions is to calculate the area in the initial image that contains the screen side features, that is, to obtain the minimum bounding rectangle of the initial image and perform edge extraction based on the minimum bounding rectangle to determine the edge area.

[0064] S205 . Split the edge region according to a preset splitting density to obtain a plurality of analysis regions.

[0065] In actual situations, the edge area is displayed on the initial image as a rectangular frame, that is, the minimum enclosing rectangle. After the minimum enclosing rectangle is expanded pixel by pixel, the terminal splits the minimum enclosing rectangle according to the preset splitting density, where each split area is an analysis area. The terminal will subsequently analyze the analysis areas one by one. All analysis areas split in this step are multiple analysis areas. The splitting density is a preset value. Setting the splitting density can obtain more points for determining the points above the side of the screen by reducing the analysis area, thereby improving the accuracy of subsequent steps.

[0066] Specifically, the splitting density can be the number of pixels, distance or other metrics, which are not limited here. The terminal splits the edge area determined according to the above steps into smaller analysis areas through the preset splitting density. All analysis areas are collectively referred to as multiple analysis areas.

[0067] S206, respectively connecting the midpoints of the two short sides of each analysis area to obtain a plurality of baselines; S207, obtaining all grayscale values ​​of all pixels on multiple baselines one by one; Steps S206 to S207 in this embodiment are similar to steps S104 to S105 in the aforementioned embodiment, and will not be described in detail here.

[0068] S208, forwardly traverse all grayscale values, and calculate the first grayscale value difference between two adjacent grayscale values ​​in all grayscale values ​​to obtain a first grayscale value difference set; Specifically, there are two traversal directions: forward traversal and reverse traversal. The forward traversal is to compare and calculate the grayscale values ​​corresponding to all pixels on all baselines using the positive direction of the image as the traversal direction. The positive direction in the initial image corresponds to the direction from bottom to top of the image. Conversely, the negative direction in the initial image is the direction from top to bottom of the image, which is the reverse traversal described in the subsequent steps.

[0069] The grayscale value difference is the difference in grayscale values ​​corresponding to two adjacent pixels. Generally, the first pixel read is the first element of the difference, and the second pixel read is the second element of the difference. The grayscale value difference is calculated as follows: grayscale value difference = grayscale value of the first element - grayscale value of the second element.

[0070] S209, obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain the first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy the change state from dark to bright; In actual situations, step S106 and step S107 are executed synchronously, that is, when the terminal calculates the grayscale value difference corresponding to a pair of two adjacent pixel points, it compares the grayscale value difference with the first preset value.

[0071] In the present application, the first preset value is actually the threshold of the grayscale value difference (grayscale amplitude), and the grayscale amplitude is the variation amplitude of the grayscale value of two adjacent pixels. Therefore, the grayscale amplitude and the grayscale value difference between two adjacent pixels are the same data. The first preset value is used to determine whether the grayscale amplitude generated when the grayscale value of two adjacent pixels changes from dark to bright can identify the grayscale value change of the pixel as an edge point. The edge point can be the edge of the actual side of the screen or the edge of an abnormal area within the side of the screen.

[0072] For details, please refer to Figure 5 , Figure 5is a schematic diagram of the initial image in this embodiment, wherein the red frame diagram is the actual state of one of the analysis areas after the minimum bounding rectangle frame is segmented by a preset density under different tilt conditions, and the red points and the green points correspond to the target points obtained by the forward traversal and the reverse traversal, respectively. In actual situations, the shooting scene is arranged according to standard rules, so that the initial image shot in the shooting scene only contains the information of the side of the screen and the background information that is almost identified as a pure color. The processed initial image will reduce the influence of noise on the image to a minimum value, and the confirmation of the minimum bounding rectangle has been completed in the aforementioned steps. The minimum bounding rectangle obtains the edge information of the side of the screen contained in the initial image by scanning, and in order to avoid information omission, the minimum bounding rectangle will be expanded by a certain number of pixels, so that when the terminal detects the initial image, each recorded grayscale amplitude change occurs within the range of the minimum bounding rectangle, and within this range, if the grayscale amplitude change of two adjacent pixel points is less than the first preset value, it can be explained that the two pixel points are considered to be edge points of a feature.

[0073] Therefore, after the terminal completes the comparison of all grayscale value differences in the first grayscale value difference set with the first preset value, the terminal will record the points of the first grayscale value difference that meet the first preset value condition. These points are obtained through forward traversal and are therefore the first forward target point set.

[0074] S210, obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Similar to the first preset value, the second preset value is used to determine whether the grayscale amplitude generated when the grayscale value of two adjacent pixels changes from light to dark can identify the grayscale value change of the pixel as an edge point, which can be the edge of the actual side of the screen or the edge of an abnormal area within the side of the screen.

[0075] After the terminal completes the comparison of all grayscale value differences in the first grayscale value difference set with the second preset value, the terminal will record the points of the first grayscale value difference that meet the second preset value condition. These points are obtained through forward traversal and are therefore the second forward target point set.

[0076] S211, reversely traverse all grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all grayscale values ​​to obtain a second grayscale value difference set; After completing the forward traversal, the terminal will reversely traverse all grayscale values. The reverse traversal is to traverse the grayscale values ​​from top to bottom of the image with the negative direction in the initial image as the direction. The order of obtaining grayscale values ​​in the reverse traversal and the forward traversal is opposite. Therefore, according to the calculation method of the grayscale value difference: grayscale value difference = grayscale value of the first element - grayscale value of the second element; it can be seen that because the order of obtaining the first element and the second element is opposite to that of the forward traversal, the results will also be different.

[0077] S212, obtaining grayscale value differences less than the first preset value in the second grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first negative target point positions, to obtain the first negative target point position collection; The judgment conditions of the grayscale value difference data obtained by the reverse traversal are the same as those of the grayscale value difference data obtained by the forward traversal, and the order of pixels obtained by the reverse traversal is opposite to that of the forward traversal. Therefore, the first negative target point collection and the first positive target point collection will not produce overlapping points, and in the two collections, the actual number of positive target points and negative target points belonging to the same analysis area can be more than one.

[0078] All elements in the second gray value difference collection are screened by the first preset value, and the corresponding pixel points in all the recorded second gray value differences are the first negative target point collection.

[0079] S213, obtaining grayscale value differences greater than a second preset value in the second grayscale value difference collection, and recording pixel points corresponding to grayscale value differences less than the second preset value as second negative target points, to obtain a second negative target point collection; All elements in the second grayscale value difference collection are screened by the second preset value, and the corresponding pixel points in all the recorded second grayscale value differences are the second negative target point collection.

[0080] In this embodiment, the grayscale value differences of adjacent grayscale value differences in the grayscale value collection are screened twice to obtain four target point collections, namely, a first positive target point collection, a first negative target point collection, a second positive target point collection, and a second negative target point collection. The grayscale value difference data in different grayscale value difference collections calculated by forward traversal and reverse traversal are compared and screened by two different preset values, respectively, to screen out suspicious points whose grayscale amplitudes suddenly change from dark to bright or suddenly change from bright to dark in the initial image. These suspicious points are target points. In actual situations, the pixel points that change from dark to bright during forward traversal and the pixel points that change from bright to dark during reverse traversal in the same analysis area may be the same points.

[0081] S214, obtaining overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; The first positive target point collection, the first negative target point collection, the second positive target point collection, and the second negative target point collection obtained through the above steps have a corresponding relationship with each other.

[0082] Specifically, the acquisition condition of the first forward target point set is the elements of the first gray value difference set that are less than the first preset value, the first gray value difference set is obtained by forward traversal, and the screening rule of the first preset value is to obtain adjacent pixel points whose gray amplitude changes from dark to bright; Similarly, the acquisition condition of the second negative target point set is the element of the second gray value difference set that is greater than the second preset value. The second gray value difference set is obtained by reverse traversal, and the screening of the second preset value is to obtain adjacent pixel points whose gray amplitude changes from bright to dark; According to Figure 5 It can be seen that in the same analysis area, the pixel points from bright to dark and the pixel points from dark to bright can obtain an overlapping pixel point under two different preset value screening conditions. This point is the overlapping point. Therefore, the terminal can filter out a group of overlapping points from the data of the first positive target point set and the second negative target point set, namely the first overlapping point.

[0083] S215, obtaining overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; Similar to the aforementioned analysis of the first positive target point collection and the second negative target point collection, because the pixel point from dark to light during forward traversal and the pixel point from light to dark during reverse traversal in the same analysis area can be the same point, the second positive target point collection and the first negative target point collection also include overlapping points, and these overlapping points are the second overlapping points.

[0084] By determining the overlapping points, the points in the target point collection can be screened. The above steps have performed two traversals of the two preset values, and the obtained target point data can correspond one to one. That is, this embodiment uses two rules (the first preset value and the second preset value) to traverse the grayscale values ​​corresponding to the pixel points on the baseline of the analysis area forward and backward, and obtains four groups of one-to-one corresponding data. The accuracy of the target points in the first overlapping point and the second overlapping point is improved by obtaining the overlapping data.

[0085] S216: Determine the outermost edge point of the screen from the first set of coincident points and the second set of coincident points according to the preset edge area.

[0086] Specifically, although the area obtained after the pixel expansion of the minimum circumscribed rectangle determined in the aforementioned step covers all the side information of the screen, the side information of the screen includes the actual contour of the original side of the screen after the expansion. Therefore, the first coincidence point collection and the second coincidence point collection also include the actual outermost edge point. According to the determination process of the aforementioned steps, the first coincidence point and the second coincidence point finally obtained by forward traversal and reverse traversal of the initial image are used to obtain the upper and lower edges of the side contour of the screen, respectively. Therefore, among the first coincidence point and the second coincidence point belonging to the same analysis area, the coincidence point with the same distance from the actual screen side parameter will be considered to be the outermost edge point of the side of the screen, that is, the outermost edge point of the screen. After determining the upper and lower edges of the screen, the left and right sides of the side area of ​​the screen are the same as the actual height of the screen. Therefore, it is determined that the distance between the coincidence points of the corresponding analysis areas in the first coincidence point collection and the second coincidence point collection meets the actual height of the actual screen. The two coincidence points will be considered to be the upper edge and lower edge of the analysis area, respectively.

[0087] The terminal will analyze the coincidence points in all analysis areas one by one according to the above method, and then obtain the outermost edge points of the screens in all analysis areas.

[0088] S217, determining the coordinate values ​​of the pixel points corresponding to the outermost edge points of the screen on the initial image, and establishing a least squares model according to the coordinate values; The terminal obtains the coordinates of the outermost edge point of the screen on the initial image. Since the height of the side of the screen is fixed, and the screen may be at an angle during actual shooting, when the coordinates of the outermost edge point are When is the independent variable, is the dependent variable.

[0089] The least squares model is established based on the above coordinates, specifically by minimizing the predicted value With observed value The sum of squared errors between and ,in is the slope, is the intercept.

[0090] S218, calculating the sum of squares of the minimized predicted value and the observed value of the least squares model, and determining the slope and intercept of the fitted target straight line; Specifically, the error square sum formula is:

[0091] From the formula of the sum of squared errors, we can know that to determine the edge of the actual detection area, the purpose of the above formula is to find m and b that minimize SSE. Therefore, after determining the formula, we need to find the partial derivatives of m and b respectively, and set the partial derivatives to 0. The derivation process is as follows:

[0092]

[0093] After sorting out the above formulas, we get the expressions of m and b, which are:

[0094]

[0095] S219: Determine the actual detection area on the initial image according to the slope and the intercept.

[0096] After obtaining the above expression, substitute the coordinate value of the outermost edge point of the screen actually obtained from the initial image to obtain the linear fitting target equation: After obtaining the linear fitting result, the lines fitted on the initial image according to the result are closed according to the parameters of the actual screen side, and the obtained edge frame is the actual outermost edge of the screen, that is, the area that actually needs to detect abnormalities.

[0097] S220, obtaining the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area, and obtaining edge points of the abnormal area.

[0098] In the above steps, the terminal generates a new screen side edge frame in the initial image based on the traversed grayscale value difference. The screen side edge frame is closer to the side edge to be detected in the actual initial image than the minimum circumscribed rectangle in the above steps, and the terminal can eliminate the interference of overlapping points outside the screen edge area and the screen overlap area on subsequent abnormal detection based on the actual detection area.

[0099] At this time, the terminal will determine the coincidence points that belong to the actual detection area based on the coordinates of all coincidence points in the first coincidence point collection and the second coincidence point collection. The actual coordinates of these coincidence points in the initial image must fall within the actual detection area.

[0100] S221, analyzing the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; Because all coordinate points in the edge points of the abnormal area must be within the actual detection area on the initial image, and the point collection of the edge points of the abnormal area is the same as the first overlapping point collection and the second overlapping point collection, the actual points are distinguished by the analysis area for different overlapping points, that is, the overlapping points must be on the baseline. Therefore, after determining the edge points of the abnormal area, the terminal will traverse all the overlapping points in the edge points of the abnormal area from left to right according to the analysis area, where adjacent points will be compared for height differences, and based on the height differences of these overlapping points, it is determined whether the overlapping points are at the same level.

[0101] S222, when the height difference is within the error range of the same level, determining that the adjacent coincident points used to calculate the height difference belong to the same level; The abnormal area is essentially also an area, so the fitting results of the upper and lower sides of the abnormal area can also be obtained in the aforementioned way, and the defect area can be obtained by closing the fitting results. Therefore, at this time, you need to analyze the height difference of the overlapping points of the two adjacent analysis areas in the terminal to distinguish the points above and below the abnormal area. The points above and the points below do not belong to the same level.

[0102] S223, determining all levels containing edge points of abnormal areas within the actual detection area; Generally, an abnormal area includes an upper edge and a lower edge. Multiple levels mean that there is more than one abnormal area in the actual detection area. According to the description of the previous steps, the overlapping points in the edge points of the abnormal area are distinguished based on the analysis area. Therefore, the terminal can determine the level of the edge points of the abnormal area according to the actual coordinate positions of the overlapping points in each analysis area.

[0103] However, in actual situations, shooting from the side of the screen may cause the side of the screen to be tilted in the image. Therefore, when determining the level of the overlapping points on the side of the screen, the terminal will determine a preset value based on the gradient direction calculated in the previous steps (the threshold is generally set to be large, but not exceeding the height of the side of the screen). When the height difference (y-axis difference) between two adjacent overlapping points meets the preset value, it is judged that the two adjacent overlapping points belong to the same level.

[0104] S224, connecting the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; After all the overlapping points of the same level are determined, all the overlapping points identified as the same level are connected. The specific connection process is to fit the upper and lower edges of the actual detection area through the aforementioned fitting steps to determine the actual position of the edge of the abnormal area.

[0105] The purpose of fitting is to determine the connection relationship between two adjacent pixels. The difference between the edge of the abnormal area and the edge of the actual detection area is that the actual detection area must be a continuous straight line, but the edge of the abnormal area can be an irregular shape. Therefore, the actual fitting process requires fitting adjacent points one by one.

[0106] After completing the fitting of the coincident points on all levels, the edge of the abnormal area is obtained.

[0107] S225. Close the edge of the abnormal region to obtain the abnormal region.

[0108] The abnormal region edge in the aforementioned step is based on the hierarchical abnormal region edge, so a step of closing the abnormal region edge is required to determine the complete abnormal region.

[0109] It should be noted that after determining the abnormal area on the side of the screen in the initial image, the terminal will input the image carrying the abnormal area information into the classification network to analyze the side abnormality, and further assist the operator in determining the cause of the abnormality. The specific analysis process is as follows: The target image with abnormal area data is input into the classification network so that the classification network analyzes the target image and outputs the cause of the abnormality.

[0110] In practical applications, to determine the cause of anomalies through a classification network, it is first necessary to collect and preprocess image data, including scaling and normalizing the image to meet the network's input requirements. After determination, a pre-trained deep learning model or a custom network structure is used to extract and analyze the image features. Through training, the model can learn the feature representation of normal images and, based on this, identify abnormal areas that are significantly different from normal features.

[0111] The classification network used in this application is a learning network that has completed basic screen side abnormality classification learning, and the classification network is directly arranged on the terminal. Therefore, after the terminal marks the abnormal area on the side of the screen, when the image with the edge of the abnormal area is input into the classification network model, the classification network model determines the abnormal type corresponding to the abnormal area of ​​the screen side image through the following analysis process: First, the terminal obtains all abnormal areas in the screen side image according to the aforementioned steps, and cuts these abnormal areas from the image according to the edges of the abnormal areas, and then inputs the cropped defective areas into the classification network for classification, and obtains the classification network outputs all classification results of the abnormalities based on the input abnormal area images, where the classification includes but is not limited to: dirt, scratches, wrinkles, and burns.

[0112] Based on the feedback from the classification network, the terminal generates a data list (including all feature parameters) according to the image ID, replaces the classification result into the data list, and finally outputs the data list data.

[0113] This embodiment is a complete process from initial image analysis to output of screen side abnormalities, which specifically describes a method for determining the actual area of ​​the screen side and a method for determining the abnormal area of ​​the screen side. After the abnormal area is determined, the terminal will also input these images into a classification model and output the abnormal cause that may correspond to the abnormal information contained in the image, thereby realizing a complete automatic detection closed loop for the abnormality of the screen side, thereby improving the efficiency and accuracy of abnormality detection on the screen side.

[0114] The above is a detailed description of the abnormality detection method for the side of the screen in the embodiment of the present application. The following is a detailed description of the abnormality detection system and device for the side of the screen.

[0115] See also Figure 3 The present application provides an embodiment of a system for detecting an abnormality on the side of a screen, the embodiment comprising: A shooting unit 301 is used to shoot an initial image of the side of the display screen; A first determining unit 302, configured to determine an edge region of the initial image; A splitting unit 303, configured to split the edge region according to a preset splitting density to obtain a plurality of analysis regions; A connecting unit 304, used for connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; An acquisition unit 305 is used to acquire all grayscale values ​​of all pixels on the plurality of reference lines one by one; A first traversal unit 306, configured to traverse all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; A first fitting unit 307, used for fitting the outermost edge point of the screen to obtain an actual detection area; A second traversal unit 308, configured to traverse all gray values ​​in the actual detection area according to the preset rule to determine edge points of the abnormal area; The second fitting unit 309 is used to fit the edge points of the abnormal region to obtain the abnormal region.

[0116] Optionally, the first traversal unit 306 is specifically used for: Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to grayscale value differences less than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; The outermost edge point of the screen is determined from the first set of coincident points and the second set of coincident points according to a preset edge area.

[0117] Optionally, the second traversal unit 308 is specifically used for: Obtain the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area to obtain edge points of the abnormal area.

[0118] Optionally, the first fitting unit 307 is specifically configured to: Determine the coordinate value of the pixel point corresponding to the outermost edge point of the screen on the initial image, and establish a least squares model according to the coordinate value; Calculate the sum of squares of the minimized predicted value and the observed value of the least squares model to determine the slope and intercept of the fitted target straight line; An actual detection area is determined on the initial image according to the slope and the intercept.

[0119] Optionally, the splitting unit 303 is specifically used for: Obtaining a minimum bounding rectangle of the initial image, and performing edge extraction based on the minimum bounding rectangle to determine an edge area; The edge region is split according to a preset splitting density to obtain a plurality of analysis regions.

[0120] Optionally, the splitting unit 303 is further configured to: Using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction, and obtain the gradient amplitude and the gradient direction; The initial image is scanned according to the gradient amplitude and the gradient direction, and pixel points satisfying the gradient amplitude on the initial image are acquired one by one and connected to obtain an edge region.

[0121] Optionally, the second fitting unit 309 is specifically configured to: Analyze the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; When the height difference is within the error range of the same level, it is determined that the adjacent coincident points used to calculate the height difference belong to the same level; Determine all levels in the actual detection area that contain edge points of the abnormal area; Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; The edge of the abnormal region is closed to obtain the abnormal region.

[0122] In this embodiment, the functions of each unit are the same as those described above. Figure 1 and Figure 2a , Figure 2b , Figure 2c The steps in the illustrated embodiments correspond to each other and will not be described again here.

[0123] See also Figure 4 The present application provides another embodiment of a device for detecting an abnormality on the side of a screen, including: Processor 401, memory 402, input and output unit 403, bus 404; The processor 401 is connected to the memory 402, the input and output unit 403 and the bus 404; Processor 401 specifically performs Figure 1 and Figure 2a , Figure 2b , Figure 2c The operations corresponding to the steps in the method are not described in detail here.

[0124] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above methods.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, etc. Various media that can store program codes.

Claims

1. A method for detecting abnormalities on the side of a screen, characterized in that: Said: Take an initial image of the side of the display; Determining an edge region of the initial image; Splitting the edge region according to a preset splitting density to obtain a plurality of analysis regions; Connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; Obtaining all grayscale values ​​of all pixels on the multiple reference lines one by one; Traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; Fitting the outermost edge points of the screen to obtain the actual detection area; Traversing all grayscale values ​​in the actual detection area according to the preset rule to determine edge points of the abnormal area; The edge points of the abnormal region are fitted to obtain the abnormal region.

2. The anomaly detection method according to claim 1, characterized in that: The traversing all the grayscale values ​​according to the preset rule to determine the outermost edge point of the screen includes: Forward traversing all the grayscale values, and calculating a first grayscale value difference between two adjacent grayscale values ​​in all the grayscale values, to obtain a first grayscale value difference set; Obtaining grayscale value differences less than a first preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences less than the first preset value as the first positive target point positions, to obtain a first positive target point position collection, wherein the first preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from dark to bright; Obtaining grayscale value differences greater than a second preset value in the first grayscale value difference collection, and recording the pixel points corresponding to the grayscale value differences greater than the second preset value as second positive target points, to obtain a second positive target point collection, wherein the second preset value is used to determine whether the pixel points corresponding to the two grayscale values ​​satisfy a change state from bright to dark; Reversely traverse all the grayscale values, and calculate the second grayscale value difference between two adjacent grayscale values ​​in all the grayscale values ​​to obtain a second grayscale value difference set; Obtaining grayscale value differences in the second grayscale value difference collection that are smaller than the first preset value, and recording pixel points corresponding to the grayscale value differences that are smaller than the first preset value as first negative target points, to obtain a first negative target point collection; Obtaining grayscale value differences greater than the second preset value in the second grayscale value difference collection, and recording pixel points corresponding to grayscale value differences less than the second preset value as second negative target points, to obtain a second negative target point collection; Acquire overlapping points of the first positive target point collection and the second negative target point collection to obtain a first overlapping point collection; Acquire overlapping points of the second positive target point collection and the first negative target point collection to obtain a second overlapping point collection; The outermost edge point of the screen is determined from the first set of coincident points and the second set of coincident points according to a preset edge area.

3. The anomaly detection method according to claim 2, characterized in that: The step of traversing all grayscale values ​​in the actual detection area according to the preset rule to determine the edge points of the abnormal area includes: Obtain the overlapping points of the first overlapping point set and the second overlapping point set in the actual detection area to obtain edge points of the abnormal area.

4. The method according to claim 1, characterized in that The step of fitting the outermost edge point of the screen to obtain the actual detection area includes: Determine the coordinate value of the pixel point corresponding to the outermost edge point of the screen on the initial image, and establish a least squares model according to the coordinate value; Calculate the sum of squares of the minimized predicted value and the observed value of the least squares model to determine the slope and intercept of the fitted target straight line; An actual detection area is determined on the initial image according to the slope and the intercept.

5. The method according to claim 1, characterized in that The edge region is split according to a preset splitting density to obtain a plurality of analysis regions, including: Obtaining a minimum bounding rectangle of the initial image, and performing edge extraction based on the minimum bounding rectangle to determine an edge area; The edge region is split according to a preset splitting density to obtain a plurality of analysis regions.

6. The method according to claim 5, characterized in that The step of obtaining a minimum bounding rectangle of the initial image and performing edge extraction according to the minimum bounding rectangle to determine an edge area includes: Using the Sobel operator to calculate the gradient of the initial image in the horizontal direction and the vertical direction, and obtain the gradient amplitude and the gradient direction; The initial image is scanned according to the gradient amplitude and the gradient direction, and pixel points satisfying the gradient amplitude on the initial image are acquired one by one and connected to obtain an edge region.

7. The method according to any one of claims 1 to 6, characterized in that The step of fitting the edge points of the abnormal region to obtain the abnormal region includes: Analyze the height differences of adjacent coincident points one by one from left to right according to the positional relationship of the edge points of the abnormal area on the initial image; When the height difference is within the error range of the same level, it is determined that the adjacent coincident points used to calculate the height difference belong to the same level; Determine all levels in the actual detection area that contain edge points of the abnormal area; Connect the edge points of the abnormal area on each level one by one to obtain the edge of the abnormal area; The edge of the abnormal region is closed to obtain the abnormal region.

8. A screen side abnormality detection system, characterized in that: The system comprises: A shooting unit, used for shooting an initial image of a side surface of the display screen; A first determining unit, configured to determine an edge region of the initial image; A splitting unit, used for splitting the edge area according to a preset splitting density to obtain a plurality of analysis areas; A connecting unit, used for connecting the midpoints of the two short sides of each of the analysis areas to obtain a plurality of reference lines; An acquisition unit, used for acquiring all grayscale values ​​of all pixels on the plurality of reference lines one by one; A first traversal unit, used for traversing all the grayscale values ​​according to a preset rule to determine the outermost edge point of the screen; A first fitting unit, used for fitting the outermost edge points of the screen to obtain an actual detection area; A second traversal unit, used to traverse all gray values ​​in the actual detection area according to the preset rule to determine the edge points of the abnormal area; The second fitting unit is used to fit the edge points of the abnormal region to obtain the abnormal region.

9. A device for detecting abnormality on the side of a screen, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 7.

Images