A display screen edge detection method, device and equipment based on deep learning

Abstract

The present invention relates to a method, device and equipment for detecting the edge of a display screen based on deep learning, the method comprising: obtaining an image to be detected for a display screen; cropping the image to be detected according to a set size to obtain at least one image block; identifying a detection result of the image to be detected based on the at least one image block through a pre-trained detection model, the detection result including at least one of an edge defect area, a cross mark area and an edge grinding area on the display screen; and measuring the display screen based on the detection result. The method of the present invention combines the detection model obtained by deep learning training to identify the image to be detected, thereby avoiding false positives and false negatives, and does not require the user to adjust the algorithm parameters, making the operation very convenient. The display screen can be measured based on the detection result through a traditional algorithm, which can take into account the respective advantages of deep learning and traditional algorithms, and well solve the edge detection task of the display screen.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, machine vision, and deep learning technologies. Specifically, the present invention relates to a display screen edge detection method, device, and equipment based on deep learning. Background Art

[0002] Machine vision is a very common technical means in industrial quality inspection. It mainly uses cameras to take pictures of industrial products, and then analyzes the product pictures through image processing technology. In this way, the product images can be observed through the actual production process to see whether there are any defects and problems in the product.

[0003] For a long time, machine vision applications relied on traditional image processing techniques to quantitatively analyze images and determine the presence of defects. Since product quality inspections in industrial production often require real-time performance, software and image processing algorithms must also be capable of real-time processing. This presents significant challenges for image processing technology, as effective and robust image processing algorithms often require complex computational processes. Consequently, the use of image processing in industrial product quality inspections has largely relied on traditional and naive image processing algorithms for defect detection. These include image data filtering and denoising, image binarization, connected component analysis, edge extraction, and morphological analysis. These techniques aim to effectively de-noise an image from noisy textures, extract the image information of interest, and then determine the content of that information. If threshold determination is required for specific inspection targets, the algorithm's output must be accompanied by numerous threshold parameters to determine which defects should be reported.

[0004] See also Figures 1 to 3 As shown, Figure 1 is the original image containing dirty defects, Figure 2 The detection results are obtained by traditional algorithms. Figure 3 is the mark position of the real defect; since the calculation logic of the traditional algorithm introduced above is relatively simple, although it can bring high computing efficiency, it will cause a large degree of misdetection in the detection results due to the simplicity of the calculation logic (see Figures 1 to 3 A large number of false positives poses a challenge to determining final product quality. A common solution is to continually modify the parameters used to detect defects. If false positives are too frequent, the parameters need to be adjusted more leniently. Conversely, if false negatives occur, the parameters need to be adjusted more strictly. While this approach can temporarily resolve the problem, it also increases dimensionality costs by constantly modifying parameters based on real-time issues.

[0005] Another solution is to use pattern recognition technology to classify all defects after identifying them as quasi-defects using image processing technology, effectively filtering out false positives. Over the past few decades, the theory and application of pattern recognition have matured significantly. The main process involves collecting training data to be identified and classified, extracting features from this data, and then training a classifier to classify unknown samples.

[0006] The biggest challenge with pattern recognition technology is the manual feature extraction and summarization required. However, humans are very limited in their ability to extract abstract features that are similar to human perception and can only extract easily quantifiable metrics. For defect classification, features are often defined as the defect's length and width, grayscale value, contrast, duty cycle, area, and so on. These features are only sufficient for classifying defects with normal morphologies and are difficult to achieve robustness. For example, the human eye can easily classify two defects with very different aspect ratios into the same category, but a machine might classify them into two different categories based on the summarized features.

[0007] Taking into account the above-mentioned problems in image processing and pattern recognition, most machine vision inspection algorithm systems adjust thresholds according to different inspection products in order to adapt to the different defect attributes of each product. This results in the edge detection algorithm in existing technologies having many parameters and inflexible engineering deployment.

[0008] Existing technologies for defect detection in industrial products rely on effective image processing of camera-captured images to segment product defects from image data. Early traditional algorithms offer very high detection timeliness, and for certain measurement tasks requiring high precision, traditional algorithm implementations and final results outperform deep learning. However, the algorithm's processing logic is overly simplistic, making it prone to missed detections and false detections. This necessitates the configuration of numerous detection parameters and various control thresholds, making the detection algorithm structure highly complex and poorly compatible with product testing. Furthermore, the operator or user must possess strong background expertise to effectively set these control thresholds. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a display screen edge detection method, device and equipment based on deep learning, aiming to solve at least one of the above technical problems.

[0010] The present invention solves the above-mentioned technical problem with the following technical solution: a display screen edge detection method based on deep learning, the method comprising:

[0011] Acquire an image to be detected for the display screen;

[0012] Cropping the image to be detected according to a set size to obtain at least one image block;

[0013] Identifying, based on the at least one image block, a detection result of the image to be detected using a pre-trained detection model, the detection result including at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen;

[0014] The display screen is measured according to the detection result.

[0015] The beneficial effects of the present invention are as follows: for images to be inspected that require edge defect detection, the detection model obtained by deep learning training can be combined to identify the images to be inspected, thereby largely avoiding false positives and missed positives. At the same time, there is no need for the user to adjust the algorithm parameters, making the operation very convenient. At the same time, in the present application, the display screen can be measured using a traditional algorithm based on the detection results, so that the respective advantages of deep learning and traditional algorithms can be taken into account at the same time, and the edge detection task of the display screen can be well solved. In addition, in the present application, the image to be inspected is cropped according to a set size to obtain at least one image block, which can facilitate the effective operation of the detection model.

[0016] On the basis of the above technical solution, the present invention can also be improved as follows.

[0017] Furthermore, the display screen is a liquid crystal glass panel display screen.

[0018] The beneficial effect of adopting the above further solution is that the solution of this application is more suitable for detecting and measuring liquid crystal glass panel display screens.

[0019] Furthermore, before cropping the image to be detected according to the set size, the method further includes:

[0020] cropping an invalid area in the image to be detected to obtain a first image excluding the invalid area, where the invalid area is an area excluding the display screen;

[0021] The above-mentioned step of cropping the image to be detected according to a set size to obtain at least one image block includes:

[0022] The first image is cropped according to a set size to obtain at least one image block.

[0023] The beneficial effect of adopting the above further solution is that the invalid area is removed from the image to be detected, which can greatly reduce the calculation area and improve the calculation efficiency.

[0024] Furthermore, before cropping the image to be detected according to the set size, the method further includes:

[0025] Vertically scaling the image to be detected according to a set compression ratio to obtain a second image;

[0026] The step of cropping the image to be detected according to a set size to obtain at least one image block includes:

[0027] The second image is cropped according to a set size to obtain at least one image block.

[0028] The beneficial effect of adopting the above-mentioned further scheme is that, for the defect detection of display screens and various precision measurement needs, there is no requirement for the vertical resolution of the image, only the horizontal resolution is required to be very high. Therefore, in the present application scheme, the image to be inspected can be vertically scaled according to the set compression ratio, thereby achieving data compression to the greatest extent, thereby saving subsequent computing efficiency.

[0029] Furthermore, the above-mentioned set size is 512*512 pixels.

[0030] The beneficial effect of adopting the above further solution is that using 512*512 pixels can avoid the image block size being too large to increase useless calculation amount, and avoid the image block size being too small to possibly fail to cover the cross mark area, resulting in inaccurate detection results.

[0031] Furthermore, if the detection result includes an edge defect area on the display screen, the step of measuring the display screen according to the detection result includes:

[0032] According to the edge defect area, the chamfer measurement of the display screen is realized;

[0033] If the detection result includes a cross mark area and an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes:

[0034] Measuring a distance from the cross mark area of ​​the display screen to an edge of the display screen according to the cross mark area and the edge grinding area;

[0035] If the detection result includes an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes:

[0036] The width of the edge grinding area of ​​the display screen is measured according to the edge grinding area.

[0037] The beneficial effect of adopting the above further solution is that different measurement requirements for the display screen can be achieved according to different detection results.

[0038] Furthermore, the method further comprises:

[0039] The detection result and the measurement result obtained by measuring the display screen are displayed.

[0040] The beneficial effect of adopting the above further solution is that the obtained measurement results and test results can be displayed, meeting the user's viewing needs for the test results and measurement results.

[0041] In a second aspect, in order to solve the above technical problems, the present invention further provides a display edge detection device based on deep learning, the device comprising:

[0042] An image acquisition module, used to acquire an image to be detected for the display screen;

[0043] A first cropping module, configured to crop the image to be detected according to a set size to obtain at least one image block;

[0044] a detection module, configured to identify, based on the at least one image block and using a pre-trained detection model, a detection result of the image to be detected, the detection result comprising at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen;

[0045] The measuring module is used to measure the display screen according to the detection result.

[0046] In a third aspect, in order to solve the above-mentioned technical problems, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a display edge detection method based on deep learning of the present application is implemented.

[0047] In a fourth aspect, in order to solve the above-mentioned technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a display screen edge detection method based on deep learning of the present application is implemented.

[0048] Additional aspects and advantages of the present application will be given in part in the following description, which will become apparent from the following description, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments of the present invention.

[0050] Figure 1 A schematic diagram of an original image containing a dirt defect in the prior art provided for one embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a detection result obtained by a traditional algorithm in the prior art provided by one embodiment of the present invention;

[0052] Figure 3 In the prior art provided for one embodiment of the present invention Figure 1 Schematic diagram of the corresponding marking position of the real defect;

[0053] Figure 4 A flowchart of a display screen edge detection method based on deep learning provided by one embodiment of the present invention;

[0054] Figure 5 A flowchart of another display screen edge detection method based on deep learning provided by one embodiment of the present invention;

[0055] Figure 6 A schematic diagram of a processing process of a detection model provided by one embodiment of the present invention;

[0056] Figure 7 A schematic diagram of a preprocessing process of an image to be detected provided by one embodiment of the present invention;

[0057] Figure 8 A schematic diagram of cropping an image to be detected into image blocks of a set size according to an embodiment of the present invention;

[0058] Figure 9 A schematic diagram of sample images, annotated images, and predicted detection results corresponding to a detection model training process provided by one embodiment of the present invention;

[0059] Figure 10 A schematic diagram of a measurement process provided by one embodiment of the present invention;

[0060] Figure 11 A schematic diagram of a straight line fitting result and two intersection points provided in one embodiment of the present invention;

[0061] Figure 12 A schematic structural diagram of a display screen edge detection device based on deep learning provided by one embodiment of the present invention;

[0062] Figure 13 The present invention provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0064] The following describes in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0065] The solution provided by the embodiments of the present invention can be applied to any application scenario requiring crack identification on a display screen. The solution provided by the embodiments of the present invention can be executed by any electronic device, for example, a user's terminal device, including at least one of the following: a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart TV, or smart in-vehicle device.

[0066] Deep learning is a new field in machine learning research. Its motivation is to build and simulate neural networks that analyze and learn data similar to the human brain. It mimics the human brain's mechanisms to interpret data, such as images, audio, and text. The essence of deep learning is to build machine learning models with many hidden layers and massive amounts of training data to learn more useful features, ultimately improving classification or prediction accuracy. Since 2012, when the AlexNet deep neural network achieved over 70% accuracy in classification on the massive ImageNet dataset, deep learning has ushered in a revolution in artificial intelligence. Deep convolutional neural networks can be used to train various models, such as segmentation, object detection, and classification. Therefore, image processing, long plagued by algorithmic robustness, can be largely replaced by deep learning. Deep learning can fit labeled data using massive samples, resulting in highly robust models that generalize well to variations in illumination, resolution, rotation, and scaling. Using deep learning for image segmentation and object detection can effectively process image data, significantly outperforming traditional rule-based image processing techniques in both stability and robustness. It can accurately locate defects and reduce the probability of false positives. With the continuous advancement of GPU technology, the speed of deep learning image processing is now fully adaptable to the real-time demands of industry. Therefore, using deep learning to solve industrial product quality inspection problems can significantly reduce development complexity while significantly improving algorithm robustness and detection accuracy.

[0067] Therefore, this application solution combines the current mainstream deep learning technology for defect detection, which greatly avoids false positives and missed positives. At the same time, there is no need for the user to adjust the algorithm parameters, making the operation very convenient. At the same time, considering the superiority of traditional algorithms, the traditional algorithms are used for fine measurement work. In this way, the respective advantages of deep learning and traditional algorithms are taken into account, and the edge detection task of LCD panels is well solved.

[0068] The long-term collection and accumulation of defect data on liquid crystal glass panels provides a large number of training samples required for deep learning. By effectively labeling the various types of defects in the training data, a robust and generalizable detection model can be trained. This trained detection model can then be used to detect defects in product images captured during actual collection. Unlike traditional algorithms, the model does not require a large number of threshold parameters for control and periodic adjustment. It also has excellent adaptability to changes in camera exposure and white balance over time, significantly reducing the difficulty of developing the algorithm system. While deep learning can largely complement or replace the application of traditional algorithms, it requires high computing resources and places high demands on data collection and labeling. Therefore, in the task of detecting edge defects in liquid crystal glass panels, the present application solution still uses traditional image processing algorithms to efficiently solve some very simple problems, ensuring that deep learning, an algorithm with high training costs, is used in the most appropriate place. This is the purpose and original intention of the present invention to combine traditional algorithms with deep learning algorithms for defect detection in liquid crystal glass panels.

[0069] Based on the above principles, the embodiment of the present invention provides a possible implementation method, such as Figure 1 As shown in FIG, a flowchart of a method for detecting the edge of a display screen based on deep learning is provided. The method can be executed by any electronic device, for example, a terminal device, or jointly executed by a terminal device and a server. For ease of description, the method provided by the embodiment of the present invention will be described below using a terminal device as an example of the execution subject. Figure 4 As shown in the flowchart, the method may include the following steps:

[0070] Step S110, obtaining an image to be detected for the display screen;

[0071] Step S120, cropping the image to be detected according to a set size to obtain at least one image block;

[0072] Step S130: identifying, based on the at least one image block, a detection result of the image to be detected using a pre-trained detection model, the detection result including at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen;

[0073] Step S140: measuring the display screen according to the detection result.

[0074] Through the method of the present invention, for images to be inspected that require edge defect detection, the detection model obtained by deep learning training can be combined to identify the images to be inspected, thereby avoiding false positives and missed positives to a great extent. At the same time, there is no need for the user to adjust the algorithm parameters, making the operation very convenient. At the same time, in the present application scheme, the display screen can be measured by a traditional algorithm based on the detection results, so that the respective advantages of deep learning and traditional algorithms can be taken into account at the same time, and the edge detection task of the display screen can be well solved. In addition, in the present application scheme, the image to be inspected is also cropped according to the set size to obtain at least one image block, which can facilitate the effective operation of the detection model.

[0075] The present invention will be further described below with reference to the following specific embodiments. Figures 4 to 6 , a display screen edge detection method based on deep learning may include the following steps:

[0076] Step S110, obtaining an image to be detected for the display screen;

[0077] The image to be detected can be an image of a display screen (corresponding to a display screen) acquired by an image acquisition device (e.g., a camera). Figure 5 The image to be inspected may or may not contain defects. Optionally, the display screen is a liquid crystal glass panel display screen. The display screen may also be a cell display screen, i.e., an ultra-thin display screen.

[0078] Since the image to be detected may contain an area that is not a display screen, the method may include: cropping an invalid area in the image to be detected to obtain a first image that does not contain the invalid area, where the invalid area is an area that does not contain the display screen.

[0079] After obtaining the first image, S120 may be directly executed. Specifically, S120 includes: cropping the first image according to a set size to obtain at least one image block. Alternatively, the first image may be further processed as follows: vertically scaling the first image according to a set compression ratio (e.g., a 50% compression ratio) to obtain a second image, i.e., compressing the first image vertically; and after obtaining the second image, S120 may be executed again. Specifically, S120 includes: cropping the second image according to the set size to obtain at least one image block.

[0080] The above process of obtaining the first image from the image to be detected and the second image from the first image can be called the preprocessing process of the image to be detected (corresponding to Figure 5 ), see also the cutting and longitudinal compression shown in Figure 7 The schematic diagram shown in Figure 7 In the figure, from left to right are the image to be detected, the first image and the second image.

[0081] It can be understood that the processing of obtaining the first image and the second image can be performed separately, that is, step S120 can be directly executed after the image to be detected is processed to obtain the first image, or step S120 can be directly executed after the image to be detected is processed to obtain the second image, both of which are within the protection scope of the present application.

[0082] After the above preprocessing, step S120 is executed to crop the image to be detected according to the set size to obtain at least one image block (image batch), corresponding to Figure 5 The vertical split shown in ;

[0083] Among them, the above-mentioned set size refers to the size of each image block, which can be set according to the input size requirements of the detection model. Optionally, the above-mentioned set size is 512*512 pixels. Using 512*512 pixels can avoid the image block size being too large to increase useless calculation amount, and avoid the image block size being too small, which may fail to cover the cross mark area, resulting in inaccurate detection results.

[0084] The specific implementation process of the above step S120 can be found in Figure 8 The schematic diagram shown, Figure 8 The left side is the image to be detected, either the first image or the second image. Figure 8 The image blocks are on the right side, and each image block has the same size.

[0085] Step S130: Based on the at least one image block, a detection result of the image to be detected is identified by a pre-trained detection model, wherein the detection result includes at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen, corresponding to Figure 5 The deep learning detection shown in the above detection results is Figure 5 The deep learning results shown in;

[0086] The above detection model is a model obtained through deep learning training. The specific training process will be described below and will not be repeated here. Figure 6 As shown, at least one image block is input into the detection model ( Figure 6 The deep learning framework shown in ), and then get Figure 6 The three different detection results on the right side of the middle are as follows: defect represents the edge defect area, mark represents the cross mark area, and grinding represents the edge grinding area.

[0087] The edge defect area specifically refers to the area on the outer edge of the screen to be inspected, that is, if the screen is rectangular, the edge refers to the edge of the rectangle; each screen needs to be cut to remove the useless area on the outermost layer of the screen. In order to measure whether the cut is too large or too small, a fixed cross mark is usually designed when designing the screen. This mark is used as a standard reference point, and the distance from this reference point to the outermost edge of the screen is measured, so that the compliance of the screen cutting can be judged by this distance. The cross mark area refers to the area on the screen where the mark image is drawn. For details, please refer to Figure 6 The edge grinding area refers to the area corresponding to the mark shown in the figure. After the cutting process is completed, there will be some unevenness on the outer edge of the screen. At this time, the outer edge of the screen will be polished with a grinding wheel. After polishing, a grinding area will be formed (after taking a picture, the grinding area will be slightly deeper than the edge before cutting). You can refer to Figure 8 The black borders at the right angles of the image.

[0088] Step S140: According to the detection result, the display screen is measured. Figure 5 The results shown in the article are based on deep learning and supplemented by traditional algorithms, including defect judgment, chamfer measurement and grinding and cutting measurement.

[0089] Optionally, the method further includes: displaying the detection results and measurement results obtained by measuring the display screen. Specifically, the display can be displayed on a user's terminal device. Alternatively, the present application solution can be implemented based on software (application) on the terminal device, and the detection results and measurement results can be displayed through a terminal interface of the software on the terminal device.

[0090] Optionally, if the detection result includes an edge defect area on the display screen, measuring the display screen according to the detection result includes:

[0091] According to the edge defect area, the chamfer measurement of the display screen is realized;

[0092] Optionally, a specific implementation method for realizing the chamfer measurement of the display screen according to the edge defect area is: according to the edge defect area, determining multiple projection points in the horizontal direction and multiple projection points in the vertical direction corresponding to the edge defect area, fitting the multiple projection points in the horizontal direction to obtain a horizontal edge (horizontal straight line), and fitting the multiple projection points in the vertical direction to obtain a vertical edge (vertical straight line), the horizontal edge and the vertical edge are perpendicular to each other, and the area formed between the horizontal edge and the vertical edge includes the chamfer area, and the intersection of the horizontal edge and the vertical edge is the vertex of the triangle (chamfer) to be fitted; using the fitting results of the horizontal edge and the vertical edge to estimate the size of the hypotenuse, and taking the length of the hypotenuse of the triangle as the result of the chamfer measurement.

[0093] If the detection result includes a cross mark area and an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes:

[0094] Measuring a distance from the cross mark area of ​​the display screen to an edge of the display screen according to the cross mark area and the edge grinding area;

[0095] Optionally, the distance from the cross mark to the panel edge is called "cutting accuracy measurement". A specific implementation method of measuring the distance from the cross mark area of ​​the display screen to the edge of the display screen based on the cross mark area and the edge grinding area is as follows:

[0096] Based on the cross mark area, a bounding rectangle of white pixels corresponding to the cross mark area is determined, a center point is calculated based on the bounding rectangle, and a distance from the cross mark area to the edge of the display screen is calculated based on the center point and the edge grinding area, that is, the distance from the center point to the edge of the edge grinding area.

[0097] If the detection result includes an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes:

[0098] The width of the edge grinding area of ​​the display screen is measured according to the edge grinding area.

[0099] Optionally, a specific implementation method of measuring the width of the edge grinding area of ​​the display screen according to the edge grinding area is as follows: determining a circumscribed rectangle corresponding to a white pixel area corresponding to the edge grinding area according to the edge grinding area, and calculating the coordinates of the center point of the circumscribed rectangle ( Figure 11 Then, perform horizontal projection of the edge grinding area on the left and right sides to obtain two vertical lines on the left and right sides (see Figure 11 ), extend the horizontal line where the center point coordinates are located, and the two intersection points between the left and right vertical lines (such as Figure 11 The distance between the first and second intersection points is the width of the edge grinding area. In addition, the distance from the center point coordinate to the second intersection point is the cutting size.

[0100] The three measurement processes mentioned above can be found in Figure 10 , Figure 10 The first row of images in the figure illustrates the chamfer measurement algorithm. The first image in the first row is a schematic diagram of the edge region. The second image shows the detection result (white area) corresponding to the edge region. The third image shows the linear fitting result of the edge region, which represents the horizontal and vertical edges in the figure. The fourth image shows the left and right intersection points of the horizontal extensions of the two fitted lines from the center of the mark. Based on these two intersection points, chamfer measurement can be performed, and the measurement result is the hypotenuse length of the triangle shown in the figure.

[0101] Figure 10 The second row of pictures in the figure describes the process of measuring the cutting accuracy and the width of the edge grinding area. The first picture in the second row is a schematic diagram of the grinding area, the second picture is a schematic diagram of the detection result of the grinding area (white area), the third picture is a schematic diagram of the fitting result of the straight line fitting of the grinding area, and the fourth picture is a schematic diagram of the left and right intersection points generated by the horizontal extension lines from the center point of the mark to the two fitted straight lines. Based on the two intersection points, the edge area and the outer edge can be measured. The distance between the two intersection points is the grinding width, and the mark to the rightmost intersection is the measurement of the cutting width of the screen.

[0102] Optionally, the above detection model can adopt the model training method in the prior art, see Figure 9 , first obtain multiple original images ( Figure 9 The first column image in the image is used as a sample image. The multiple sample images contain edge defect areas, cross mark areas, and edge grinding areas. Then, for each original image, the edge defect areas, cross mark areas, and edge grinding areas are marked to obtain multiple labeled images ( Figure 9 As an example, take the images containing edge defect area, cross mark area and edge grinding area as an example, the annotated images can be referred to Figure 9 The second column of images in the image can be marked with black marks. Professional marking tools can be used for marking.

[0103] Then, based on each original image and each annotated image, the deep learning model is trained to obtain the prediction detection results corresponding to each original image (for details, please refer to Figure 9 In the third column of images in the figure, each predicted detection result (respectively, the cross mark area, the edge grinding area, and the edge defect area) corresponds to a predicted detection area. Then, according to each labeled image and each predicted detection result, the loss value of the deep learning model can be obtained. When the loss value meets the training end condition, the deep learning model that meets the training end condition is used as the detection model. If the loss value does not meet the training end condition, the model parameters of the deep learning model can be adjusted, and the deep learning model can be retrained based on the adjusted model parameters until the loss value meets the training end condition.

[0104] pass Figure 9 As can be seen from the image corresponding to the detection results in the third column, deep learning detection does not cause false positives and is very accurate in locating the detection target.

[0105] Optionally, the above-mentioned deep learning model may be a BiSeNet-V2 (Bilateral Network with Guided Aggregation for Real Semantic Segmentation) segmentation model.

[0106] The solution of this application fully considers the various difficulties that arise in cell edge detection technology. First, LCD panel cell detection requires algorithm software to achieve very high timeliness, so as to ensure the normal mass production of products in the assembly line. After performing multiple preprocessing steps on the original acquired image, this solution significantly saves expensive deep learning computing resources and reduces detection time by more than 60%. Secondly, the intervention of deep learning can ensure the accuracy of defect detection and ensure that the false alarm rate is minimized. This can achieve the highest confidence level for detected defects, minimize the probability of manual re-judgment, and improve production efficiency. The detection rate of defects using deep learning reaches more than 98%, and the false alarm rate is only about 3%, which is far superior to traditional algorithms. Finally, the detection area given by deep learning can provide effective regional orientation for precise measurement work. In these areas, traditional algorithms are used again for precise measurement work. Compared with directly using traditional algorithms for regional detection and measurement, better preliminary positioning results can be obtained, avoiding the low robustness and excessive parameter settings brought by direct use of traditional algorithms in measurement.

[0107] This paper introduces an edge detection technology for TFT-LCD (liquid crystal glass panel display) in all processes based on a hybrid processing of traditional algorithms and deep learning algorithms. This technology retains some advantages of traditional image processing algorithms, while using deep learning technology to make up for some shortcomings of traditional algorithms. Combining the two technologies to solve edge detection problems has a more stable effect, fewer parameters, and more flexible engineering deployment.

[0108] Based on Figure 1 Based on the same principle as the method shown in , the embodiment of the present invention also provides a display screen edge detection device 20 based on deep learning, such as Figure 12 As shown in , the display screen edge detection device 20 based on deep learning may include an image acquisition module 210, a first cropping module 220, a detection module 230 and a measurement module 240, wherein:

[0109] An image acquisition module 210 is used to acquire an image to be detected for the display screen;

[0110] A first cropping module 220 is configured to crop the image to be detected according to a set size to obtain at least one image block;

[0111] a detection module 230 configured to identify, based on the at least one image block and using a pre-trained detection model, a detection result of the image to be detected, the detection result comprising at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen;

[0112] The measuring module 240 is configured to measure the display screen according to the detection result.

[0113] Optionally, the display screen is a liquid crystal glass panel display screen.

[0114] Optionally, before cropping the image to be detected according to the set size, the method further includes:

[0115] a second cropping module, configured to crop an invalid area in the image to be detected to obtain a first image excluding the invalid area, wherein the invalid area is an area excluding the display screen;

[0116] When the first cropping module 220 crops the image to be detected according to a set size to obtain at least one image block, it is specifically configured to:

[0117] The first image is cropped according to a set size to obtain at least one image block.

[0118] Optionally, before cropping the image to be detected according to a set size, the device further includes:

[0119] A scaling module, configured to vertically scale the image to be detected according to a set compression ratio to obtain a second image;

[0120] When the first cropping module 220 crops the image to be detected according to a set size to obtain at least one image block, it is specifically configured to:

[0121] The second image is cropped according to a set size to obtain at least one image block.

[0122] Optionally, the above set size is 512*512 pixels.

[0123] Optionally, if the detection result includes an edge defect area on the display screen, the measurement module 240 is specifically configured to:

[0124] According to the edge defect area, the chamfer measurement of the display screen is realized;

[0125] If the detection result includes a cross mark area and an edge grinding area on the display screen, the measurement module 240 is specifically configured to:

[0126] Measuring a distance from the cross mark area of ​​the display screen to an edge of the display screen according to the cross mark area and the edge grinding area;

[0127] If the detection result includes an edge grinding area on the display screen, the measurement module 240, when measuring the display screen according to the detection result, is specifically used to: measure the width of the edge grinding area of ​​the display screen according to the edge grinding area.

[0128] Optionally, the device further includes:

[0129] The display module is used to display the detection result and the measurement result obtained by measuring the display screen.

[0130] The deep learning-based display edge detection device of an embodiment of the present invention can execute the deep learning-based display edge detection method provided by the embodiment of the present invention. The implementation principles are similar. The actions performed by each module and unit in the deep learning-based display edge detection device in each embodiment of the present invention correspond to the steps in the deep learning-based display edge detection method in each embodiment of the present invention. For the detailed functional description of each module of the deep learning-based display edge detection device, please refer to the description of the corresponding deep learning-based display edge detection method shown in the previous text, and will not be repeated here.

[0131] Among them, the above-mentioned deep learning-based display edge detection device can be a computer program (including program code) running in a computer device, for example, the deep learning-based display edge detection device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present invention.

[0132] In some embodiments, the deep learning-based display edge detection device provided by the embodiments of the present invention can be implemented using a combination of software and hardware. As an example, the deep learning-based display edge detection device provided by the embodiments of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the deep learning-based display edge detection method provided by the embodiments of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0133] In other embodiments, the display screen edge detection device based on deep learning provided by the embodiments of the present invention can be implemented in software. Figure 12 A deep learning-based display edge detection device stored in a memory is shown. The device can be software in the form of a program or plug-in, and includes a series of modules, including an image acquisition module 210, a first cropping module 220, a detection module 230, and a measurement module 240, for implementing the deep learning-based display edge detection method provided in an embodiment of the present invention.

[0134] The modules involved in the embodiments of the present invention may be implemented in software or hardware, wherein the name of a module does not necessarily limit the module itself.

[0135] Based on the same principle as the method shown in the embodiments of the present invention, an electronic device is also provided in the embodiments of the present invention, which may include but is not limited to: a processor and a memory; the memory is used to store computer programs; the processor is used to execute the method shown in any embodiment of the present invention by calling the computer program.

[0136] In an alternative embodiment, an electronic device is provided, such as Figure 13 As shown, Figure 13 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0137] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0138] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0139] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0140] The memory 4003 is used to store application code (computer program) for executing the solution of the present invention, and is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0141] Among them, the electronic device can also be a terminal device, Figure 13 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0142] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment.

[0143] According to another aspect of the present invention, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the deep learning-based display edge detection method provided in the various implementations described above.

[0144] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0145] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0146] The computer-readable storage medium provided by the embodiments of the present invention may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0147] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0148] The above description is merely a preferred embodiment of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

Claims

1. A display screen edge detection method based on deep learning, characterized in that: The following steps are involved: Acquire an image to be detected for the display screen; Cropping the image to be detected according to a set size to obtain at least one image block; Identifying, based on the at least one image block, a detection result of the image to be detected using a pre-trained detection model, the detection result including at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen; According to the detection result, the display screen is measured; If the detection result includes a cross mark area and an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes: Determine, based on the cross mark area, a bounding rectangle of white pixels corresponding to the cross mark area, calculate a center point based on the bounding rectangle, and calculate a distance from the cross mark area to the edge of the display screen based on the center point and the edge grinding area; If the detection result includes an edge grinding area on the display screen, the step of measuring the display screen according to the detection result includes: According to the edge grinding area, the circumscribed rectangle corresponding to the white pixel area corresponding to the edge grinding area is determined, the center point coordinates of the circumscribed rectangle are calculated, and the edge grinding area is horizontally projected to the left and right sides to obtain two left and right longitudinal vertical lines. The horizontal straight line where the center point coordinates are located is extended, and the distance between the two intersection points of the horizontal straight line and the left and right longitudinal vertical lines is determined as the width of the edge grinding area.

2. The method according to claim 1, characterized in that The display screen is a liquid crystal glass panel display screen.

3. The method according to claim 1, characterized in that Before cropping the image to be detected according to the set size, the method further includes: cropping an invalid area in the image to be detected to obtain a first image excluding the invalid area, where the invalid area is an area excluding the display screen; The step of cropping the image to be detected according to a set size to obtain at least one image block includes: The first image is cropped according to a set size to obtain at least one image block.

4. The method according to claim 1, wherein Before cropping the image to be detected according to the set size, the method further includes: Vertically scaling the image to be detected according to a set compression ratio to obtain a second image; The second image is cropped according to a set size to obtain at least one image block.

5. The method according to any one of claims 1 to 4, characterized in that The set size is 512*512 pixels.

6. The method according to any one of claims 1 to 4, characterized in that If the detection result includes an edge defect area on the display screen, the step of measuring the display screen according to the detection result includes: The chamfer measurement of the display screen is achieved according to the edge defect area.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The detection result and the measurement result obtained by measuring the display screen are displayed.

8. A display screen edge detection device based on deep learning, characterized in that: The display screen edge detection method based on deep learning according to claim 1 is adopted, and the device includes: An image acquisition module, used to acquire an image to be detected for the display screen; A first cropping module, configured to crop the image to be detected according to a set size to obtain at least one image block; a detection module, configured to identify, based on the at least one image block and using a pre-trained detection model, a detection result of the image to be detected, the detection result comprising at least one of an edge defect area, a cross mark area, and an edge grinding area on the display screen; The measuring module is used to measure the display screen according to the detection result.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images