Method and related product for detecting broken lines of a display screen

Abstract

The present application discloses a method and related products for detecting broken lines of a display screen. The method includes: obtaining a display screen image when the display screen border is lit; determining the position of the border line to be detected in the display screen image, where the position includes the minimum circumscribed rotated rectangle; extracting the detection region image and the region of interest image of the border line to be detected from the display screen image according to the minimum circumscribed rotated rectangle; and performing broken line detection on the border line to be detected according to the detection region image and the region of interest image. The above method of the present application can significantly improve the accuracy of detecting defects of the display screen border line.

Description

Technical Field

[0001] This application generally relates to the technical field of display screen detection. More specifically, this application relates to a method, a processing device, and a computer-readable storage medium for detecting broken lines on a display screen. Background Art

[0002] In the manufacturing and quality inspection processes of VR (Virtual Reality) display screens, the integrity of the display screen border line (Waku line) is directly related to the display function and reliability of the screen. As a key conductive line distributed around the screen edge, the Waku line undertakes important functions such as signal transmission, power supply, and electromagnetic shielding. However, due to the complexity of the manufacturing process, material property limitations, and the influence of mechanical stress during subsequent use, the Waku line is prone to defects such as breaks or missing parts. These defects have three significant characteristics: first, the defect location must be in the screen edge area; second, the break location is random and may occur at any part of the edge; third, the break length is uncertain, and there may be microcracks at the micron level or obvious breaks at the millimeter level.

[0003] Currently, the industry's detection of the Waku line mainly relies on the Waku screen detection technology. This technology generally operates in two ways: one is to accurately input a specific pulse current into the screen drive circuit to activate only the screen edge area in a short time; the other is to send a high-contrast lighting instruction to the pixel units at the screen edge by means of a customized image signal, so as to achieve the effect of activating only the screen edge area. This special lighting method greatly enhances the detection pertinence. However, in the actual application process, this technology has certain problems in terms of accuracy. For example, the detection accuracy is easily affected by ambient light interference, resulting in deviations in the detection results of Waku line defects under different lighting conditions and being unable to accurately judge the true situation of the defects. In addition, in the face of a complex screen structure, it is difficult to accurately identify the details at the screen edge, and it is easy to miss or misdetect defects such as microcracks at the micron level or some breaks with more concealed positions, thus affecting the accuracy of the detection of Waku line defects.

[0004] In view of this, there is an urgent need to provide a method, a processing device, and a computer-readable storage medium for detecting broken lines on a display screen, so as to improve the detection accuracy of Waku broken lines, etc. Summary of the Invention

[0005] In order to solve at least one or more of the above-mentioned technical problems, this application proposes a method, a processing device, and a computer-readable storage medium for detecting broken lines on a display screen in multiple aspects.

[0006] In a first aspect, the present application provides a method for detecting broken lines of a display screen, including: obtaining a display screen image when the border of the display screen is lit; determining the position of a border line to be detected in the display screen image, where the position includes a minimum circumscribed rotated rectangle; extracting a detection region image and a region of interest image of the border line to be detected from the display screen image according to the minimum circumscribed rotated rectangle; and performing broken line detection on the border line to be detected according to the detection region image and the region of interest image.

[0007] In some embodiments, the determining the position of the border line to be detected in the display screen image includes: performing projective transformation on the display screen image by using a transformation matrix; performing binarization processing on the display screen image after projective transformation, where the display screen region and the non-display screen region in the display screen image correspond to different grayscales respectively; performing edge detection in different directions on the binarized display screen image to obtain border line region images in different directions; determining the minimum circumscribed rotated rectangle of the border line to be detected in the border line region images; and performing inverse projective transformation on the minimum circumscribed rotated rectangle of the border line to be detected in the border line region images by using the transformation matrix to obtain the minimum circumscribed rotated rectangle of the border line to be detected in the display screen image.

[0008] In some embodiments, the determining the minimum circumscribed rotated rectangle of the border line to be detected in the border line region images includes: determining the minimum circumscribed rotated rectangle of each border line in the border line region images; and identifying the border line to be detected and the minimum circumscribed rotated rectangle of the border line to be detected according to the center coordinates of the minimum circumscribed rotated rectangle of the border line.

[0009] In some embodiments, the extracting the detection region image of the border line to be detected from the display screen image according to the minimum circumscribed rotated rectangle includes: determining a target transformation matrix for projecting the minimum circumscribed rotated rectangle to a preset horizontal rectangle, where the sorting order of the corner coordinates of the minimum circumscribed rotated rectangle is the same as that of the preset horizontal rectangle; and extracting the horizontal detection region image and the region of interest image of the border line to be detected from the display screen image according to the target transformation matrix and the minimum circumscribed rotated rectangle.

[0010] In some embodiments, the extracting the horizontal detection region image of the border line to be detected from the display screen image according to the target transformation matrix and the minimum circumscribed rotated rectangle includes: performing projective transformation on the region where the minimum circumscribed rotated rectangle of the border line to be detected in the display screen image is located by using the target transformation matrix to obtain the horizontal detection region image of the border line to be detected.

[0011] In some embodiments, the extracting, from the display screen image, the horizontal region of interest image of the border line to be detected according to the target transformation matrix and the minimum bounding rotated rectangle includes: performing a projective transformation on the display screen image by using the transformation matrix; performing binarization processing on the display screen image after the projective transformation, where the regions where the display screen is located and the regions where the non-display screen is located in the display screen image correspond to different grayscales; performing an inverse projective transformation on the binarized display screen image by using the transformation matrix; and performing a projective transformation on the region where the minimum bounding rotated rectangle of the border line to be detected is located in the display screen image after the inverse projective transformation by using the target transformation matrix to obtain the horizontal region of interest image of the border line to be detected.

[0012] In some embodiments, before detecting the broken line of the border line to be detected according to the detection region image and the region of interest image, it includes: aligning the border line to be detected in the detection region image with the center of the region of interest in the region of interest image; and cropping the non-overlapping parts of the detection region image and the region of interest image; or cropping the lower part of the detection region image and the upper part of the region of interest image with a preset height.

[0013] In some embodiments, before detecting the broken line of the border line to be detected according to the detection region image and the region of interest image, it further includes: performing Gaussian filtering on the detection region image; and performing adaptive threshold segmentation on the detection region image.

[0014] In some embodiments, the detecting the broken line of the border line to be detected according to the detection region image and the region of interest image includes: constructing a detection list of the border line to be detected according to the detection region image; constructing an interest list of the border line to be detected according to the region of interest image; marking the number of pixel points with a gray value greater than 0 in each column of the detection list, where the number of pixel points represents the line width of each column; determining the gray value of the corresponding column in the interest list for each column in the detection list; and determining the column with the number of pixel points being the preset pixel point number threshold and the gray value of the corresponding column in the interest list being the target gray value in the detection list as the broken line position.

[0015] In some embodiments, after determining the column with the number of pixel points being the preset pixel point number threshold and the gray value of the corresponding column in the interest list being the target gray value in the detection list as the broken line position, it includes: performing an inverse projective transformation on the broken line position by using the target transformation matrix to obtain the corresponding position of the broken line position in the display screen image; and making a mark at the corresponding position in the display screen image.

[0016] In a second aspect, the present application provides a processing device, including: a processor configured to execute program instructions; and a memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to execute the method described in the above first aspect and any of its embodiments.

[0017] In a third aspect, the present application provides a computer-readable storage medium storing program instructions, characterized in that, when the program instructions are loaded and executed by a processor, the processor is caused to execute the method described in the above first aspect and any of its embodiments.

[0018] Through the method for detecting a broken line of a display screen provided as above, by acquiring a display screen image when the border of the display screen is lit and determining the position of the border line to be detected in the display screen image, the position including the minimum circumscribed rotated rectangle. Then, according to the minimum circumscribed rotated rectangle, a detection region image and a region of interest image of the border line to be detected are extracted from the display screen image. Thus, based on the detection region image and the region of interest image, a broken line detection is performed on the border line to be detected. The detection scheme according to the embodiments of the present application can significantly improve the accuracy of detecting defects of the display screen border line. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present application will become readily understandable. In the drawings, several embodiments of the present application are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0020] Figure 1 An exemplary flowchart of steps of a method for detecting a broken line of a display screen showing some embodiments of the present application;

[0021] Figure 2 An exemplary flowchart of steps of a method for detecting a broken line of a display screen showing some embodiments of the present application;

[0022] Figure 3 An exemplary flowchart of steps of a method for detecting a broken line of a display screen showing some embodiments of the present application;

[0023] Figure 4 An exemplary flowchart of steps of a method for detecting a broken line of a display screen showing some embodiments of the present application;

[0024] Figure 5 An exemplary flowchart of steps of a method for detecting a broken line of a display screen showing some embodiments of the present application;

[0025] Figure 6Exemplary flowchart of steps of a method for detecting broken lines in a display screen according to some embodiments of the present application;

[0026] Figure 7 Exemplary schematic diagram of an image obtained after performing corresponding steps of a method for detecting broken lines in a display screen according to some embodiments of the present application;

[0027] Figure 8 Exemplary schematic diagram of an image obtained after performing corresponding steps of a method for detecting broken lines in a display screen according to some embodiments of the present application;

[0028] Figure 9 Exemplary structural block diagram of a processing device according to some embodiments of the present application. Detailed implementation manners

[0029] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0030] It should be understood that the terms "including" and "comprising" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0031] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and claims of the present application, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be further understood that the term " / and / " used in the specification and claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] As used in this specification and the claims, the term "if" can be interpreted as "when...", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0033] The following will describe in detail the specific embodiments of the present application with reference to the accompanying drawings.

[0034] Exemplary application scenarios

[0035] In the process of display screen production and manufacturing, during the production line, detecting the open circuit of each display screen can promptly identify products with open circuit problems, effectively prevent unqualified products from entering the market, and significantly improve the product yield rate and production efficiency. In terms of display screen repair services, when a display screen has faults such as abnormal display, maintenance personnel can use effective detection methods to quickly and accurately locate the open circuit position, thereby carrying out targeted repairs, which not only shortens the repair time but also reduces the repair cost. In the display screen quality monitoring scenario, for large screens in public places and industrial control display screens that are used for a long time, regularly conducting open circuit detection can promptly detect potential open circuit problems, and carry out maintenance and replacement work in advance to ensure the normal operation of the display screen and prevent information display errors or equipment failures caused by open circuit faults. The display screen described here can be a VR display screen or other types of display screens.

[0036] Among them, in the production and use of display screens, the stability of the Waku line is crucial. However, at present, there is a significant problem of low accuracy in the detection of the Waku line of display screens. This problem seriously affects the overall quality and performance of display screens.

[0037] Exemplary implementation solutions

[0038] In view of this, the present application provides a method for detecting open circuits in display screens. It obtains the display screen image when the display screen border is lit, and determines the position of the border line to be detected in the display screen image, and this position includes the minimum circumscribed rotated rectangle. Then, according to the minimum circumscribed rotated rectangle, the detection area image and the region of interest image of the border line to be detected are extracted from the display screen image. Thus, based on the detection area image and the region of interest image, open circuit detection is performed on the border line to be detected. Through the detection scheme of the embodiments of the present application, the accuracy of detecting defects in the display screen border line can be significantly improved.

[0039] Figure 1 Shows method 100 for detecting open circuits in display screens according to some embodiments of the present application, which includes steps S101 to S104. As Figure 1 shown, first, at step S101, the display screen image when the display screen border is lit is obtained. In this embodiment, when only the border of the display screen is lit and the rest is pure black, image acquisition is performed on it, so as to obtain a display screen image with clear border lines and a pure black background as shown in Figure 7 Figure (a).

[0040] In some embodiments, to create a scenario where only the border of the display screen is lit and the rest appears pure black, the display screen is placed in a dark environment, and the display screen is controlled to display a Waku screen to light up the border. At this time, the display screen is imaged by a camera, and a display screen image with clear border lines and a pure black background can be obtained.

[0041] It can be understood that the Waku screen is a special state of the display screen image, which refers to the image presented when only the border part of the display screen is lit and the other areas of the screen are in the off state. In this image, the observer can only see the edge border of the display screen being lit, showing clear border lines, while the central and other non-border areas of the screen are pure black, without any image or light display.

[0042] In the application scenarios of some embodiments, the display screen is placed on a stage. Once the border line is lit, the light will be reflected onto the stage. Affected by this, the display screen image captured by the camera will inevitably include some areas of the stage. Therefore, after obtaining the display screen image, the gray level of the part of the stage area in the display screen image can be replaced with 0, corresponding to black, so as to eliminate potential interference and avoid adverse effects on the subsequent detection process.

[0043] In step S102, the position of the border line to be detected in the display screen image is determined. In this embodiment, the position of the border line to be detected can be represented by the minimum circumscribed rotated rectangle. The minimum circumscribed rotated rectangle refers to a rectangle that can completely enclose the border line to be detected and has the smallest area among all possible directions. This rectangle can be rotated at any angle without being aligned with the coordinate axes. Thus, it can be understood that the minimum circumscribed rotated rectangle can most compactly represent the position of the border line to be detected.

[0044] It should be noted that the border line to be detected here is at least one border line in the display screen image. For example, the border line to be detected can be all the border lines in the display screen image, that is, all the borders of the display screen, or one or several border lines in the display screen image, that is, one or several borders of the display screen. Those skilled in the art can select the corresponding border line as the border line to be detected according to the actual detection requirements, so as to perform disconnection detection on it.

[0045] In step S103, the detection area image and the region of interest image of the border line to be detected are extracted from the display screen image according to the minimum circumscribed rotated rectangle. In this embodiment, the detection area image is directly extracted from the display screen image captured by the camera, completely retaining the original pixel details of the border line to be detected, and providing comprehensive and rich visual features for subsequent analysis of the border line to be detected. The region of interest image is extracted after performing binary processing on the display screen image captured by the camera, and can efficiently eliminate the interference of background noise irrelevant to the display screen.

[0046] It can be understood that during the binarization process, the grayscale of the non - region of interest and the region of interest in the display screen image is simplified into two states: black and white. Among them, the region of interest corresponds to the display screen area, and the non - region of interest corresponds to the non - display screen area. As an example, the grayscale of the display screen area in the display screen image can be set to 255, corresponding to white, and the grayscale of the non - display screen area can be set to 0, corresponding to black.

[0047] In step S104, based on the detected region image and the region of interest image, break - line detection is performed on the border line to be detected. In some embodiments, full - image break - line detection can be performed on the detected region image, and while detecting, determine the grayscale of the current detection position in the detected region image corresponding to the same position in the region of interest image. This grayscale can reflect whether the current detection position in the detected region image is the region of interest. When the full - image break - line detection of the detected region image is completed, all possible break - line positions of the border line to be detected are obtained. After the full - image break - line detection of the detected region image, a series of possible break - line positions of the border line to be detected will be obtained. However, not all of these possible break - line positions truly correspond to the break - line situation of the border line to be detected. Some of them may be false detection results caused by interference factors such as background noise, uneven illumination, and image acquisition device errors.

[0048] In order to accurately identify the break - line positions that truly belong to the border line to be detected from these possible break - line positions, a key judgment mechanism needs to be introduced, that is, to judge whether the grayscale of each break - line position corresponding to the same position in the region of interest image is the target grayscale. The target grayscale defined here refers to the grayscale of the region of interest in the region of interest image. For example, if the grayscale of the region of interest in the region of interest image is 255, then the target grayscale is set to 255 at this time.

[0049] For each possible break - line position, if the grayscale of the corresponding same position in the region of interest image is the target grayscale, then determine that this break - line position is the actual break - line position of the border line to be detected. On the contrary, if the grayscale of this break - line position is not the target grayscale, it indicates that this break - line position is not the real break - line position of the border line to be detected, but a false detection result caused by the above - mentioned background or other interference factors, and it should be excluded from the set of possible break - line positions.

[0050] Through this rigorous and targeted screening process, the finally retained break - line positions have a high degree of credibility, effectively excluding the influence of various interference factors, and can more accurately and reliably reflect the real break - line situation of the border line to be detected.

[0051] The above is combined overall Figure 1Method 100 for detecting broken lines of a display screen according to some embodiments of the present application, which obtains a display screen image when the border of the display screen is lit and determines the position of the border line to be detected in the display screen image, and this position includes the minimum circumscribed rotated rectangle. Then, according to the minimum circumscribed rotated rectangle, the detection region image and the region of interest image of the border line to be detected are extracted from the display screen image. Thus, according to the detection region image and the region of interest image, broken line detection is performed on the border line to be detected. Through the detection scheme of the embodiments of the present application, the accuracy of broken line detection of the display screen border line can be significantly improved.

[0052] However, those skilled in the art can know that the above Figure 1 shown method 100 is exemplary rather than restrictive, and those skilled in the art can make flexible adjustments.

[0053] Next, in combination with Figure 2 Method 200 for detecting broken lines of a display screen according to some embodiments of the present application is shown. This method 200 is a specific implementation manner of step S102 in the above method 100, but does not limit it. Method 200 includes steps S201 to S205.

[0054] First, in step S201, a projection transformation is performed on the display screen image using a transformation matrix. In this embodiment, a projection transformation is performed on the display screen in the display screen image collected by the camera using the transformation matrix to correct the display screen in the display screen image, thereby eliminating the geometric deformation caused by the shooting angle or lens distortion of the display screen image, and making the border line of the display screen in a standard position, which helps to improve the detection accuracy and simplify the analysis process.

[0055] Specifically, the specific execution process of step S201 includes: locating the four outermost sides of the display screen in the display screen image collected by the camera; then calculating the coordinates of the four intersection points where the four sides intersect pairwise, and these intersection point coordinates are the corner point coordinates; finally, performing a projection transformation on the four corner point coordinates in the display screen image using the transformation matrix, thereby realizing the correction of the display screen in the display screen image. As Figure 7 Figure (a) is the display screen image originally collected by the camera, and the bright lines in the figure are the border lines of the display screen. Figure 7 Figure (b) is the display screen image after extracting the corner point coordinates, and corner0~3 in the figure are the corner point coordinates. It should be noted that Figure 7 The dotted square formed along the four outermost sides of the display screen in (b) is drawn to clearly show the position of the corner point coordinates, and in reality, this dotted square may or may not be displayed. Figure 7 Figure (c) is the display screen image after performing a projection transformation using the transformation matrix, and the display screen in this display screen image has been corrected. It should be noted that Figure 7 (a)- Figure 7The image shown in (r) is the image obtained after performing the corresponding steps in the display screen disconnection detection method of some embodiments of the present application on the display screen image of an octagon.

[0056] In some embodiments, an edge detection algorithm can be used to locate the four outermost sides of the display screen in the display screen image originally collected by the camera, or a deep learning saliency detection model can be used to locate the four outermost sides of the display screen in the display screen image. This embodiment does not make specific limitations on this, and those skilled in the art can make flexible selections or adopt other positioning methods.

[0057] In some embodiments, the transformation matrix can adopt trans (Perspective Transformation Matrix, perspective transformation matrix).

[0058] Then in step S202, the display screen image after projective transformation is binarized, where the display screen area and the non-display screen area in the display screen image correspond to different grayscales. In this embodiment, based on the above description, it can be known that the display screen area is the region of interest, and the non-display screen area is the non-region of interest. Setting these two areas to different grayscales can significantly enhance the visual recognition of the region of interest and effectively highlight it from the entire picture. At the same time, it greatly weakens the visual presence of the non-region of interest, thereby eliminating the possible interference of the non-region of interest in the subsequent disconnection detection link and ensuring the accuracy and reliability of the detection result.

[0059] For example, in some embodiments, it can be like Figure 7 The grayscale of the display screen area in the display screen image after projective transformation shown in (c) is set to 255, corresponding to white, and the grayscale of the non-display screen area in the display screen image is set to 0, corresponding to black, so as to obtain the display screen image after binarization shown in Figure 7 (d).

[0060] Further in step S203, edge detection in different directions is performed on the binarized display screen image to obtain border line area images in different directions.

[0061] The shapes of display screens are diverse, and they can generally be divided into two types: right-angled polygon display screens and non-right-angled polygon display screens. It can be understood that a right-angled polygon display screen refers to a polygon in which all sides are composed of vertical sides and horizontal sides and there are no hypotenuse sides, such as a square and a rectangle. A non-right-angled polygon display screen refers to a polygon that has hypotenuse sides in addition to vertical sides and horizontal sides, such as the Figure 7 octagon display screen shown in (a). Next, specific implementation manners for extracting border line area images in different directions for these two types of display screens are given respectively, but no specific limitations are imposed on them.

[0062] If the display screen is a rectangular polygon, the Soble operator is used to perform vertical edge detection on the display screen image to obtain a vertical border line area image. The Soble operator is also used to perform horizontal edge detection on the display screen image to obtain a horizontal border line area image. Since rectangular polygon displays have no oblique edges, the vertical border line area image obtained here only includes the vertical border lines of the display screen, and the horizontal border line area image also only includes the horizontal border lines of the display screen. Therefore, both vertical and horizontal border line area images can be successfully extracted.

[0063] If the display screen is a non-right-angled polygon, the Soble operator is also used to perform edge detection on the display screen image in the vertical direction. However, due to the presence of the oblique edge, a border line area image including the vertical direction and the oblique direction will eventually be obtained. In other words, the border line area image includes vertical border lines and oblique border lines. And the Soble operator is used to perform edge detection on the display screen image in the horizontal direction. Similarly, due to the presence of the oblique edge, a border line area image including the horizontal direction and the oblique direction will eventually be obtained. In other words, the border line area image includes horizontal border lines and oblique border lines. In order to obtain the border line area images of the display screen in the vertical direction, horizontal direction and oblique direction respectively, the above-mentioned border line area images including the vertical direction and the oblique direction and the border line area image including the horizontal direction and the oblique direction will be further processed.

[0064] To obtain an oblique border line region image, an AND operation is performed on the aforementioned vertical and oblique border line region images and the horizontal and oblique border line region images to obtain an oblique border line region image. The oblique border line region image only includes oblique border lines.

[0065] In order to obtain a vertical border line area image, a morphological opening operation is performed on the aforementioned border line area image including vertical and oblique directions, thereby removing the oblique border lines in the image and obtaining a vertical border line area image including only vertical edges.

[0066] Similarly, to obtain a horizontal border line area image, a morphological opening operation is performed on the aforementioned border line area image including horizontal and oblique directions, thereby removing the oblique border lines in the image and obtaining a horizontal border line area image including only horizontal edges.

[0067] Finally, in step S204 , the minimum circumscribed rotated rectangle of each to-be-detected border line in the border line region image is determined.

[0068] In this embodiment, it is necessary to first determine the minimum circumscribed rotated rectangle of each border line in the border line region image. As a feasible implementation, the edge contours of each border line in the border line region image can be searched first, and then the minimum circumscribed rotated rectangle of the edge contours can be calculated. As an example, the cv2.findContours function provided in the OpenCV library of Python can be used to search for the edge contours of each border line to be detected in the border line region image, and then the cv2.minAreaRect function provided in the OpenCV library of Python can be used to calculate the minimum circumscribed rotated rectangle of each edge contour.

[0069] However, since the border line region image is a sub-image separated from the display screen image, the positions of its border lines have shifted compared to the display screen image. Therefore, the positions of the same border line in the border line region image and the display screen image are different. So, after obtaining the minimum circumscribed rotated rectangles of each border line in the border line region image, it is also necessary to further identify the border lines to be detected and the minimum circumscribed rotated rectangles of the border lines to be detected according to the center coordinates of the minimum circumscribed rotated rectangles of each border line. It can be understood that the border lines to be detected are at least one border line in the display screen image. Therefore, here it is actually to identify which specific border line in the display screen each border line in the border line region image corresponds to.

[0070] In some embodiments, for example Figure 7The octagonal display screen shown in (a). For the image of the border line area in the vertical direction, the left vertical border line and the right vertical border line can be identified by comparing the x values of the midpoint coordinates of the minimum circumscribed rotated rectangle. Among them, the x value of the left vertical border line is the smallest, and the x value of the right vertical border line is the largest. For the image of the border line area in the horizontal direction, the upper horizontal border line and the lower horizontal border line can be identified by comparing the y values of the center coordinates of the minimum circumscribed rotated rectangle. Among them, the y value of the upper horizontal border line is the smallest, and the y value of the lower horizontal border line is the largest. For the image of the border line area in the inclined direction, the upper left inclined border line, the lower left inclined border line, the upper right inclined border line, and the lower right inclined border line can be identified according to the relationship between the center coordinates of the minimum circumscribed rotated rectangle and the center coordinates of the border line area image. Among them, the center coordinates of the minimum circumscribed rotated rectangle are expressed as (x1, y1), and the center coordinates of the border line area image are expressed as (x2, y2). The relationship between the center coordinates of each inclined border line and the center coordinates of the border line area image is that for the upper left inclined border line, x1 < x2 and y1 < y2; for the upper right inclined border line, x1 > x2 and y1 < y2; for the lower left inclined border line, x1 < x2 and y1 > y2; for the lower right inclined border line, x1 > x2 and y1 > y2. Thus, through the above relationship between the center coordinates of the minimum circumscribed rotated rectangle of each inclined border line and the center coordinates of the border line area image, each inclined border line and its corresponding minimum circumscribed rotated rectangle can be identified. Thus, through the above method, each detected border line and its corresponding minimum circumscribed rotated rectangle in the border line area image can be identified.

[0071] In some embodiments, after identifying each detected border line and its corresponding minimum circumscribed rotated rectangle in the border line area image, the detected border lines in each border line area image can be sorted or numbered according to a predetermined rule, so as to quickly find the corresponding detected border line and its corresponding minimum circumscribed rotated rectangle, and be able to flexibly and accurately control which detected border line to perform defect detection on during subsequent defect detection.

[0072] As an example, for the vertical border line and the horizontal border line, they can be sorted according to the rule of the upper horizontal border line, the right vertical border line, and the lower left vertical border line. For the inclined border line, they can be sorted according to the rule of the upper right inclined border line, the lower right inclined border line, the lower left inclined border line, and the upper left inclined border line.

[0073] In step S205, an inverse projective transformation is performed on the minimum circumscribed rotated rectangle of each border line to be detected in the border line region image by using a transformation matrix, so as to obtain the minimum circumscribed rotated rectangle of each border line to be detected in the display screen image. In this embodiment, it can be understood that the border line region image is an image obtained by straightening the display screen image by using the transformation matrix. Therefore, by performing an inverse projective transformation on the minimum circumscribed rotated rectangle of each border line to be detected in the border line region image, the corresponding minimum circumscribed rotated rectangle in the display screen image can be obtained.

[0074] Therefore, in the embodiment of the present application, a projective transformation is performed on the display screen image, then the minimum circumscribed rotated rectangle of the border line to be detected is obtained from the projected image, and finally an inverse projective transformation is performed to obtain the corresponding minimum circumscribed rotated rectangle in the display screen image, which ingeniously simplifies the complex graphic processing process, effectively reduces the amount of calculation, and improves the operation efficiency.

[0075] In some embodiments, after performing an inverse projective transformation on the minimum circumscribed rotated rectangle of each border line to be detected in the border line region image, some errors may occur, resulting in the minimum circumscribed rotated rectangle of the border line to be detected in the obtained display screen image being unable to completely enclose the border line to be detected. Therefore, before performing step S205, the minimum circumscribed rotated rectangle of each border line to be detected in the border line region image can be expanded, that is, the width and height of the minimum circumscribed rotated rectangle are both increased by an expansion amount to enlarge it. Furthermore, after performing step S205, a minimum circumscribed rotated rectangle that completely encloses the border line to be detected in the display screen image can be obtained.

[0076] In some embodiments, after obtaining the minimum circumscribed rotated rectangle of the border line to be detected in the display screen image by performing step S205, the minimum circumscribed rotated rectangle of the border line to be detected in the display screen is expanded so that the expanded minimum circumscribed rotated rectangle can completely enclose the border line to be detected. Those skilled in the art can flexibly select a method, and this embodiment does not make specific limitations in this regard.

[0077] As described above in combination with Figure 2Method 200 for detecting broken lines of a display screen according to some embodiments of the present application is further described. By using a transformation matrix to perform a projective transformation on the display screen image and performing binarization processing on the display screen image after the projective transformation, where the display screen area and the non-display screen area in the display screen image correspond to different gray levels respectively. Then, edge detection in different directions is performed on the binarized display screen image to obtain border line area images in different directions. And the minimum circumscribed rotated rectangle of each border line to be detected in the border line area image is determined. Finally, by using the transformation matrix to perform an inverse projective transformation on the minimum circumscribed rotated rectangle of each border line to be detected in the border line area image, the minimum circumscribed rotated rectangle of each border line to be detected in the display screen image is obtained. By using the above method to determine the minimum circumscribed rotated rectangle of each border line to be detected in the display screen image in the embodiments of the present application, the complex graphic processing process can be significantly simplified, the calculation amount can be effectively reduced, and the operation efficiency can be improved.

[0078] However, those skilled in the art can know that the above Figure 2 shown method 200 is exemplary rather than restrictive, and those skilled in the art can make flexible adjustments.

[0079] Next, in combination with Figure 3 Method 300 for detecting broken lines of a display screen according to some embodiments of the present application is shown. This method 300 is a specific implementation manner of step S103 in the foregoing method 100, but does not limit it. Method 300 includes step S301 and step S302.

[0080] First, in step S301, a target transformation matrix for projecting the minimum circumscribed rotated rectangle to a preset horizontal rectangle is determined, where the sorting order of the corner point coordinates of the minimum circumscribed rotated rectangle is the same as that of the preset horizontal rectangle. In this embodiment, the preset horizontal rectangle is pre-created, and its four corner point coordinates are (0, 0), (boxLen, 0), (boxLen, shortSide), and (0, shortSide) respectively. To enable the subsequent conversion of the border lines to be detected in the display screen image to be horizontal, the corner point coordinates of the minimum circumscribed rotated rectangle of the border lines to be detected in the display screen image are sorted according to the sorting order of the corner point coordinates of the above preset horizontal rectangle, so that the corner point coordinates of the two corresponding to each other one by one. That is, the corner point coordinates of the minimum circumscribed rotated rectangle are sorted clockwise as the upper left corner, the upper right corner, the lower right corner, and the lower left corner. After that, the target transformation matrix for projecting the minimum circumscribed rotated rectangle to the preset horizontal rectangle can be determined.

[0081] Furthermore, in step S302, according to the target transformation matrix and the minimum circumscribed rotated rectangle, a horizontal detection area image and a region of interest image of the border lines to be detected are extracted from the display screen image.

[0082] In this embodiment, the target transformation matrix is directly used to perform a projective transformation on the region where the minimum circumscribed rotated rectangle of the border line to be detected is located in the display screen image collected by the camera, so as to obtain the detection region image of the horizontal direction of the border line to be detected. As Figure 7 (a) is the display screen image collected by the camera, Figure 7 (i) is Figure 7 the detection region image of the horizontal direction of the left vertical border line in (a).

[0083] The extraction of the region of interest image includes the following steps: First, the projective transformation is performed on the display screen image by using the transformation matrix. Then, the display screen image after the projective transformation is binarized, where the region where the display screen is located and the region where the non-display screen is located in the display screen image correspond to different grayscales. The above two steps are the same as step S201 and step S202 of the previous method 200 respectively. For the relevant details, reference can be made to the corresponding content in the previous text, and no further elaboration will be given here. After that, the inverse projective transformation is performed on the binarized display screen image by using the transformation matrix. Thus, the binarization processing of the display screen image is realized, and the background noise interference irrelevant to the display screen can be efficiently removed. Finally, the target transformation matrix is used to perform a projective transformation on the region where the minimum circumscribed rotated rectangle of the border line to be detected is located in the display screen image after the inverse projective transformation, so as to obtain the region of interest image of the horizontal direction of the border line to be detected. As Figure 7 (a) is the display screen image collected by the camera, Figure 7 (c) is the display screen image after the projective transformation, Figure 7 (d) is Figure 7 the display screen image after the binarization processing of the display screen image shown in (c), Figure 7 (e) is Figure 7 the display screen image after the inverse projective transformation of the display screen image shown in (d). It can be seen that Figure 7 the display screen image shown in (e) and Figure 7 the original display screen image collected by the camera shown in (a) only have changes in grayscale, Figure 7 (j) is extracted from Figure 7 (e) as the region of interest image of the horizontal direction of the left vertical border line. As can be seen from Figure 7 (i), the detection region image completely retains the original pixel details of the border line to be detected, providing comprehensive and rich visual features for the subsequent analysis of the border line to be detected. As can be seen from Figure 7 (j), the region of interest image efficiently removes the background noise interference irrelevant to the display screen, which is beneficial to improving the accuracy of defect detection.

[0084] The above is combined with Figure 3Method 300 for detecting broken lines in a display screen according to some embodiments of the present application is further described. It determines a target transformation matrix for transforming the minimum circumscribed rotated rectangle of the border line to be detected in the display screen image into a preset horizontal rectangle, where the sorting order of the corner coordinates of the minimum circumscribed rotated rectangle is the same as that of the preset horizontal rectangle. Then, according to the target transformation matrix and the minimum circumscribed rotated rectangle, the horizontal detection region image and the region of interest image of the border line to be detected are successfully extracted from the display screen image. Through the above method, not only the background noise and interference from other border lines unrelated to the border line to be detected are removed, but also the horizontal image of the border line to be detected can be extracted, facilitating subsequent defect detection.

[0085] However, as those skilled in the art can know, the method 300 shown above is exemplary rather than restrictive, and those skilled in the art can make flexible adjustments. Figure 3 The method 300 shown above is exemplary rather than restrictive, and those skilled in the art can make flexible adjustments.

[0086] Next, in combination with Figure 4 Method 400 for detecting broken lines in a display screen according to some embodiments of the present application is described. This method 400 is executed before step S104 of the above method 100 and includes step S401 and step S402.

[0087] In step S401, the border line to be detected in the detection region image is aligned with the center of the region of interest in the region of interest image.

[0088] In step S402, the non-overlapping parts of the detection region image and the region of interest image are cropped.

[0089] In this embodiment, since the border line to be detected is the edge part of the display screen, the border line to be detected in the detection region image is located at the upper edge of the region of interest in the region of interest image. However, due to the fact that the edges of the region of interest are vulnerable to factors such as uneven illumination and image distortion, some edge regions may be too bright to cover up defects or too dark to blur features, which will increase the detection difficulty and cause false positives and missed detections, affecting the detection accuracy and reliability. Therefore, in order to improve the reliability of broken line detection, the border line to be detected in the detection region image can be included inside the region of interest in the region of interest image through the above alignment and cropping operations. In addition, the useless regions in the detection region image are cropped, which can also reduce the computational amount of defect detection and improve the detection efficiency.

[0090] It should be noted that it is not required that the border line to be detected in the detection region image must be aligned with the center of the region of interest in the region of interest image, as long as the condition that the border line to be detected is completely contained in the region of interest is satisfied.

[0091] In some embodiments, since the border line to be detected in the detection area image is located at the edge of the region of interest in the region-of-interest image, after aligning the border line to be detected in the detection area image with the region of interest in the region-of-interest image, the lower part of the detection area image will surely be in an independent spatial position and will not overlap with the region-of-interest image, and the upper part of the region-of-interest image will also surely be in an independent spatial position and will not overlap with the detection area image. Therefore, in some embodiments, steps S401 and S402 described above may not be executed, and the following step S403 may be directly executed.

[0092] In step S403, the lower part of the detection area image and the upper part of the region of interest are cropped at a preset height. This can also make the border line to be detected in the detection area image be included inside the region of interest in the region-of-interest image. It can be understood that the cropping height of both is the same and is the preset height, so the sizes of the detection area image and the region-of-interest image are the same after cropping. It should be noted that the preset height of cropping should ensure that the border line to be detected in the detection area image and the region of interest in the region-of-interest image are not cropped.

[0093] As Figure 7 (k) is a partial enlarged view obtained by cropping the detection area image shown in Figure 7 (i), and it only shows the right half of the detection area image after cropping. Figure 7 (l) is a partial enlarged image obtained by cropping the region-of-interest image shown in Figure 7 (j), and it only shows the right half of the region-of-interest image after cropping. It can be seen that the region of interest in the white range in the region-of-interest image moves upward as a whole, and the border line to be detected in the detection area image moves downward as a whole. At this time, the border line to be detected in the detection area image is completely included inside the region of interest in the region-of-interest image.

[0094] The above Figure 4 has specifically described the method 400 for detecting broken lines on a display screen according to some embodiments of the present application. By aligning the border line to be detected in the detection area image with the center of the region of interest in the region-of-interest image and cropping the non-overlapping parts of the detection area image and the region-of-interest image after alignment, the border line to be detected in the detection area image is included inside the region of interest in the region-of-interest image, thereby improving the reliability and accuracy of subsequent defect detection based on the detection area image and the region-of-interest image.

[0095] However, those skilled in the art can know that the method 400 shown above Figure 4 is exemplary rather than restrictive, and those skilled in the art can make flexible adjustments.

[0096] In the display screen image acquisition stage, the limitations of environmental lighting and the instability of the lighting state of the display screen itself have a significant impact on the image quality of the finally acquired display screen image. This impact is manifested as obvious differences in the gray levels of the border lines at different edges of the display screen. When the detection area image is directly extracted from this display screen image, this gray level difference problem also continues to the detection area image. At the same time, the detection area image is also interfered by noise. The existence of noise further blurs the image information, and the inconsistency of the border line gray levels makes the image features more complex. In the subsequent detection process, such an image condition of the detection area image is very likely to cause problems of over-detection or missed detection, resulting in a significant reduction in the accuracy of the detection result.

[0097] To solve the above defects, next, in combination with Figure 5 A method 500 for detecting broken lines on a display screen according to some embodiments of the present application will be described. This method 500 can be executed before step S104 of the foregoing method 100, and it includes step S501 and step S502.

[0098] In step S501, Gaussian filtering is performed on the detection area image. In this embodiment, performing Gaussian filtering on the detection area image can smooth the image information of the detection area image, thereby removing image noise.

[0099] In step S502, adaptive threshold segmentation is performed on the detection area image. In this embodiment, performing adaptive threshold segmentation on the detection area image can effectively overcome the defect of uneven segmentation caused by the gray level difference of the border lines, avoid the occurrence of over-detection or missed detection, and thus improve the accuracy and reliability of the broken line detection result.

[0100] Assume that the width of the detection area image is W and the height is H. The execution process of the above Gaussian filtering and adaptive threshold segmentation of the detection image is as follows: First, the detection area image is divided into W different windows with a width of 1 and a height of H; Gaussian filtering is performed on each window; the image gray level mean value of each window after Gaussian filtering is determined; a preset threshold offset compensation value is obtained, and the difference between the image gray level mean value of each window and the preset threshold offset compensation value is calculated, and this difference is the segmentation threshold; the image gray level of each window is modified to this difference. Thus, the detection area image after Gaussian filtering and adaptive threshold segmentation is obtained. Since the segmentation threshold will be slightly higher than the image gray level mean value, the preset threshold offset compensation value is usually set to a negative value. For example Figure 7 (m), which is the detection area image after Gaussian filtering and adaptive threshold segmentation, it can be clearly seen that the gray level contrast of the border line to be detected is more distinct, and the line contour is more prominent and clear compared with that before processing.

[0101] The above in combination with Figure 5A specific description is given of a method 500 for detecting broken lines on a display screen in some embodiments of the present application. The method 500 improves the clarity and recognizability of the border lines to be detected in the detection area image by performing Gaussian filtering on the detection area image and performing adaptive threshold segmentation on the detection area image, which helps to improve the reliability and accuracy of subsequent broken line detection.

[0102] Next, combine Figure 6 The method 600 for detecting display screen disconnection according to some embodiments of the present application is described. The method 600 is a specific implementation of step S104 in the above method 100, but is not intended to limit the same. The method 600 includes steps S601 to S605.

[0103] In step S601, a detection list of the border lines to be detected is constructed based on the detection area image. In this embodiment, the border lines to be detected in the detection area image are horizontal. Therefore, the detection area image is divided into multiple equally spaced sections along the horizontal dimension, and each section is treated as a column to construct the detection list. This method can carefully analyze the local features of the border lines to be detected, providing an accurate data foundation for subsequent detection work.

[0104] In step S602, an interest list of border lines to be detected is constructed based on the ROI image. In this embodiment, the ROI in the ROI image is also horizontally oriented. A method consistent with the above is adopted to divide the ROI image into multiple equally spaced sections along the horizontal dimension, with each section corresponding to a column, thereby constructing an interest list.

[0105] It's important to note that the spacing between columns in the test list and the list of interest remains the same. This ensures that columns in the same position in the test list and the list of interest have a spatial correspondence, enabling efficient information matching and comparison between the two, greatly improving the coordination and accuracy of the entire testing process.

[0106] In some embodiments, to construct an interest list, a list with the same width and height as the ROI image and a grayscale of 0 can be created. The width range of defect detection is then determined based on the distance between the border line to be detected and both sides of the detection area image. For example, if the distance between the border line to be detected and the left side of the detection area image is a and the distance between the right side is b, then the width range of defect detection is w-(a+b). A determination is then made as to whether an ROI exists within this width range in the ROI image. If so, the grayscale at the same position in the list is set to 255. The interest list thus constructed corresponds completely to the detection list in terms of spatial position.

[0107] like Figure 7(n) is the constructed list of images of interest. This Figure 7 (n) is not currently magnified and the list content cannot be directly observed. However, the list actually exists objectively in the image. If you need to know the form of the list, you can refer to Figure 7 (q) and Figure 7 (r) for the magnification effect. However, it should be noted that Figure 7 (q) and Figure 7 (r) the list in which the number of pixels is marked, while currently Figure 7 the list in (n) has not marked this content, and the relevant annotation instructions will be explained in detail in the following content.

[0108] In the image of the detection area, there are corners and corners in the border line to be detected, but these are not part of the border line to be detected. In order to more accurately extract the key information of the border line to be detected and avoid the interference caused by corners and corners, in some embodiments, morphological erosion operations are performed on the list of images of interest. The main purpose of the morphological erosion operation is to shrink the objects in the image. Here, the gray values at the corners and corners are changed to 0, corresponding to black, so as to achieve the effect of removing corners and corners.

[0109] Such as Figure 7 (o) is the image of the list of images of interest after removing corners and corners. It can be seen that compared with Figure 7 (n), the gray values of the areas on the left and right sides of the area of interest that belong to the corners and corners become 0, corresponding to black. Similarly, this Figure 7 (o) is not currently magnified and the list content cannot be directly observed. However, the list actually exists objectively in the image. If you need to know the form of the list, you can refer to Figure 7 (q) and Figure 7 (r) for the magnification effect. However, it should be noted that Figure 7 (q) and Figure 7 (r) the list in which the number of pixels is marked, while currently Figure 7 the list in (o) has not marked this content, and the relevant annotation instructions will be explained in detail in the following content.

[0110] In step S603, the number of pixel points with a gray value greater than 0 is marked for each column in the detection list, where the number of pixel points represents the line width of each column. In this embodiment, the detection list represents the line width of the border line to be detected corresponding to each column by the number of pixel points with a gray value greater than 0 marked for each column. Under normal circumstances, this method can effectively reflect the line width information. Because if there is a break defect in the border line to be detected at a certain position, the number of pixel points marked for the corresponding column at that position will be significantly reduced, and even drop to 0 in extreme cases. However, in addition to the pixel points generated by the border line to be detected, there are also pixel points generated by other interference factors in the detection list. These interference factors may stem from noise during the image acquisition process, the influence of the surrounding environment, or interference from other non-border line features of the image itself. For this reason, simply based on the number of pixel points in a certain column, it is impossible to accurately determine whether the pixel point distribution corresponding to this column is that of the border line to be detected or the pixel point distribution caused by other interference factors. Therefore, the following steps S604 and S605 need to be continued.

[0111] In step S604, the gray value of the corresponding column at the same position in the interest list is determined for each column in the detection list. In this embodiment, this step can be executed synchronously with the above step S403, that is, when the number of pixel points with a gray value greater than 0 is marked for each column in the detection list, the gray value of the corresponding column at the same position in the interest list is determined synchronously. Of course, this step can also be executed after the above step S603 is executed, and this embodiment does not make specific limitations on this. The gray value of the corresponding column at the same position in the interest list for each column in the detection list can determine whether the pixel points of this column are marked for the border line to be detected. Through these two values, it is possible to accurately screen out which are the pixel point marks belonging to the border line to be detected and which are the interfering pixel point marks, so as to accurately identify the break position of the border line to be detected.

[0112] Such as Figure 7 (q) is Figure 7 A partial enlarged view after marking the number of pixel points with a gray value greater than 0 for the left red frame area in the (p) detection list, Figure 7 (r) is Figure 7 A partial enlarged view after marking the number of pixel points with a gray value greater than 0 for the right red frame area in the (p) detection list. It can be seen that the number of pixel points in each column of this detection list is accurately matched with the line width of the area where it is located, showing a high degree of consistency.

[0113] Finally, in step S605, the columns in the detection list with the number of pixel points being the preset pixel point number threshold and the gray level of the corresponding same-position column in the region of interest list being the target gray level are determined as the wire break positions. In this embodiment, the preset pixel point number threshold is 0, and the target gray level is the gray level of the region of interest in the region-of-interest image. Based on the foregoing description, it can be known that the gray level of the region of interest is 255. It can be understood that the line width corresponding to the column with the number of pixel points being 0 in the detection list is 0. Therefore, the column with the number of pixel points greater than 0 in the detection list being 0 and the gray level of the corresponding same-position column in the region of interest list being 255 is the true wire break position of the border line to be detected. Thus, the wire break detection of the border line to be detected can be accurately achieved.

[0114] In some embodiments, after obtaining the wire break position of the border line to be detected as described above, an inverse projection transformation is further performed on the wire break position using the target transformation matrix to obtain its corresponding position in the original display screen image collected by the camera. Then, a mark is made at this corresponding position in the display screen image. Thus, the mark of the wire break position of the border line to be detected in the original display screen image is achieved.

[0115] Considering that the width of each column is relatively narrow, if the marking is performed column by column, not only will the calculation amount be significantly increased, but also when there are adjacent columns that are both wire breaks, the marking effect on the display screen image will not look very beautiful. Therefore, to solve the above defects, in some embodiments, all adjacent columns in the detection list with the number of pixel points being the preset pixel point number threshold and the gray level of the corresponding same-position column in the region of interest list being the target gray level can be counted. Then, the overall position where these adjacent columns are located is obtained. After that, an inverse projection transformation is performed on this overall position using the target transformation matrix to obtain its corresponding position in the original display screen image collected by the camera. Finally, a mark is made at this position in the display screen image. The marking method can be to enclose this position in the display screen image with a highlighted rectangular frame so that the user can quickly and clearly see the wire break position. As Figure 8 shown in the display screen image, the overall positions of the adjacent wire break positions in each border line to be detected in this display screen image are all marked with highlighted red rectangular frames.

[0116] Further, in some embodiments, after or while making a mark on the original display screen image, the wire break length can be further marked. Specifically, the number of adjacent columns in the detection list with the number of pixel points being the preset pixel point number threshold and the gray level of the corresponding same-position column in the region of interest list being the target gray level can be calculated. And according to the number of adjacent columns and the actual physical size corresponding to each column, the actual length of the wire break is calculated. Then, this actual length is marked at or near the corresponding wire break position in the display screen image, so as to provide more intuitive and quantitative information for subsequent analysis and help relevant personnel more comprehensively understand the severity of the wire break defect.

[0117] In some embodiments, after calculating the actual length of the broken line, it is further possible to determine whether the actual length is greater than a preset length threshold. If it is greater than the preset length threshold, the adjacent column is determined as the broken line position. If it is less than the preset length threshold, it indicates that the length of the broken line is very short and can be ignored, and the adjacent column is not determined as the broken line position.

[0118] The above combination Figure 6 Specific descriptions have been made for the method 600 for detecting broken lines in a display screen according to some embodiments of the present application. It constructs a detection list of the border lines to be detected based on the image of the detection area, and constructs an interest list of the border lines to be detected based on the image of the region of interest. Then, the number of pixel points with a gray value greater than 0 is marked for each column in the detection list, where the number of pixel points represents the line width of each column, and the gray value of the corresponding column at the same position in the interest list is determined for each column in the detection list. Finally, the column with the number of pixel points being the preset pixel point number threshold and the gray value of the corresponding column at the same position in the interest list being the target gray value in the detection list is determined as the broken line position. Through this method, all broken line positions of the border lines to be detected can be accurately identified, improving the reliability and accuracy of broken line detection.

[0119] Exemplary device

[0120] After introducing the method of the embodiments of the present application, the embodiments of the present application also provide an exemplary structural block diagram of a processing device for the method for detecting broken lines in a display screen as Figure 9 shown.

[0121] As Figure 9 shown, the processing device 900 of the present application may include a processor 901 and a memory 902. Among them, an executable program is stored on the memory 902, and the processor 901 can load and execute the executable program, so that the processing device 900 implements any method step described above.

[0122] In an example scenario, the processor 901 can be used to control the memory 902. Further, the processor 901 can be a central processing unit (CPU), an application processor (AP), etc. integrated in the processing device 900; and the memory 902, as the hardware for implementing the storage function, can be a read-only memory (ROM), a dynamic RAM (DRAM), etc.

[0123] The embodiments of the present application also provide a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed by a processor of an electronic device, the processor is caused to execute at least part of the steps of the above method of the present application. In the embodiments of the present application, there is also provided a computer program product, including a computer program or instruction, which, when executed by a processor, implements the method described in the embodiments of the present application.

[0124] The computer-readable storage medium provided by the embodiments of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0125] The above computer-readable storage medium may be embedded or externally connected to the processing device for the method of detecting a broken line of a display screen.

[0126] Although multiple embodiments of the present application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many changes, alterations, and alternative ways may occur to those skilled in the art without departing from the spirit and scope of the present application. It should be understood that various alternative embodiments of the embodiments of the present application described herein may be adopted in the practice of the present application. The appended claims are intended to define the scope of protection of the present application and thus cover equivalents or alternative solutions within the scope of these claims.

Claims

1. A method for detecting broken lines of a display screen, characterized in that Including: Obtaining a display screen image when the border of the display screen is lit; Determining the position of the border line to be detected in the display screen image, where the position includes the minimum circumscribed rotated rectangle; Extracting the detection region image and the region of interest image of the border line to be detected from the display screen image according to the minimum circumscribed rotated rectangle; And Performing a break line detection on the border line to be detected according to the detection region image and the region of interest image, which includes: Constructing a detection list of the border line to be detected according to the detection region image; Constructing an interest list of the border line to be detected according to the region of interest image; Marking the number of pixel points with a gray value greater than 0 in each column of the detection list, where the number of pixel points characterizes the line width of each column; Determining the gray level of the corresponding same-position column in the interest list for each column in the detection list; Determining the columns in the detection list where the number of pixel points is the preset pixel point number threshold and the gray level of the corresponding same-position column in the interest list is the target gray level as the break line positions; Performing an inverse projection transformation on the break line positions using the target transformation matrix to obtain the corresponding positions of the break line positions in the display screen image; Performing an identification at the corresponding positions in the display screen image.

2. The method according to claim 1, wherein The determining the position of the border line to be detected in the display screen image includes: Performing a projection transformation on the display screen image using the transformation matrix; Performing a binarization process on the display screen image after the projection transformation, where the display screen region and the non-display screen region in the display screen image correspond to different gray levels; Performing edge detections in different directions on the binarized display screen image to obtain border line region images in different directions; Determining the minimum circumscribed rotated rectangle of the border line to be detected in the border line region image; and Performing an inverse projection transformation on the minimum circumscribed rotated rectangle of the border line to be detected in the border line region image using the transformation matrix to obtain the minimum circumscribed rotated rectangle of the border line to be detected in the display screen image.

3. The method according to claim 2, wherein The determining the minimum circumscribed rotated rectangle of the border line to be detected in the border line region image includes: Determining the minimum circumscribed rotated rectangles of the respective border lines in the border line region image; and Identifying the border line to be detected and the minimum circumscribed rotated rectangle of the border line to be detected according to the center coordinates of the minimum circumscribed rotated rectangles of the border lines.

4. The method according to claim 1, wherein The extracting the detection region image of the border line to be detected from the display screen image according to the minimum circumscribed rotated rectangle includes: Determining the target transformation matrix for projecting the minimum circumscribed rotated rectangle to a preset horizontal rectangle, where the corner point coordinate sorting orders of the minimum circumscribed rotated rectangle and the preset horizontal rectangle are the same; and Extracting the horizontal detection region image and the region of interest image of the border line to be detected from the display screen image according to the target transformation matrix and the minimum circumscribed rotated rectangle.

5. The method according to claim 4, wherein The extracting the horizontal detection region image of the border line to be detected from the display screen image according to the target transformation matrix and the minimum circumscribed rotated rectangle includes: Perform a projective transformation on the region where the minimum circumscribed rotated rectangle of the to-be-detected border line in the display screen image using the target transformation matrix to obtain the image of the detection region in the horizontal direction of the to-be-detected border line.

6. The method according to claim 4, characterized in that, The extracting the image of the region of interest in the horizontal direction of the to-be-detected border line from the display screen image according to the target transformation matrix and the minimum circumscribed rotated rectangle includes: Perform a projective transformation on the display screen image using the transformation matrix; Perform binarization processing on the display screen image after the projective transformation, where the regions where the display screen is located and the regions where the non-display screen is located in the display screen image correspond to different grayscales; Perform an inverse projective transformation on the binarized display screen image using the transformation matrix; and Perform a projective transformation on the region where the minimum circumscribed rotated rectangle of the to-be-detected border line in the display screen image after the inverse projective transformation using the target transformation matrix to obtain the image of the region of interest in the horizontal direction of the to-be-detected border line.

7. The method according to claim 1, characterized in that, Before performing the break line detection on the to-be-detected border line according to the detection region image and the region of interest image, it includes: Align the to-be-detected border line in the detection region image with the center of the region of interest in the region of interest image; and Crop the non-overlapping parts of the detection region image and the region of interest image; Or, Crop the lower part of the detection region image and the upper part of the region of interest image with a preset height.

8. The method according to claim 1, characterized in that Before performing the break line detection on the to-be-detected border line according to the detection region image and the region of interest image, it further includes: Perform Gaussian filtering on the detection region image; and Perform adaptive threshold segmentation on the detection region image.

9. A processing device, characterized in that, It includes: A processor configured to execute program instructions; And A memory configured to store program instructions, which when loaded and executed by the processor, cause the processor to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are loaded and executed by the processor, cause the processor to execute the method according to any one of claims 1 to 8.

Images