Display screen peripheral circuit detection method, device, electronic equipment and storage medium
The defect detection model trained by the full convolution network algorithm detects the peripheral circuit of the display screen, which solves the problems of low detection accuracy and poor system performance in the prior art, and realizes high-precision and high-scaling circuit quality detection.
Patent Information
- Application Number
- CN201810709408.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-07-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2038-07-02
AI Technical Summary
The existing peripheral circuit detection methods of display screens are subjectively affected by people, resulting in low detection accuracy, poor system performance and low business expansion capabilities.
The defect detection model trained by the full convolutional network (FCN) algorithm is used to preprocess the image of the display peripheral circuit, and then defect detection is performed. The image acquisition device is used to collect images in real time and input them into the detection model. The circuit is determined based on the detection results. Quality, optimized by sending alarms and control instructions through the controller.
It improves the classification accuracy and intelligence capabilities of defect detection, improves system performance and business scalability capabilities, and solves the subjective impact of manual detection.
Smart Images

Figure CN109087281B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to defect detection technology, and in particular to a display screen peripheral circuit detection method, device, electronic equipment and storage medium. Background Art
[0002] With the development of science and technology, the role of information display technology in people's lives is increasing day by day. Display screens are also widely used due to their small size, light weight, low power, high resolution, high brightness and no geometric deformation. However, in the production process of display screens, defects may exist in the peripheral circuits of the display screen due to process and environmental reasons, such as point defects, foreign body defects and scratch defects. Therefore, the detection of the peripheral circuit of the display screen is an important link in the production process.
[0003] In the prior art, the peripheral circuit detection of display screens mainly adopts manual detection or machine-assisted manual detection methods. Specifically, the manual detection method refers to relying on industry experts to observe the pictures collected from the production environment and make judgments with their naked eyes; the machine-assisted manual detection method refers to first using a quality inspection system that has solidified the experience of industry experts to detect the peripheral circuit images of the display screen to be inspected, preliminarily screening out pictures suspected of having defects, and then having industry experts manually detect and judge the pictures suspected of having defects.
[0004] However, both manual detection methods and machine-assisted manual detection methods are greatly influenced by human subjective factors, resulting in low detection accuracy, poor system performance, and low business expansion capabilities. Summary of the invention
[0005] The present invention provides a display screen peripheral circuit detection method, device, electronic device and storage medium to overcome the problems of low detection accuracy, poor system performance and low business expansion capability in the existing display screen peripheral circuit defect detection method due to the large influence of human subjective factors.
[0006] According to a first aspect of the present invention, a display screen peripheral circuit detection method is provided, comprising:
[0007] Receiving a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image acquired by an image acquisition device on the display screen peripheral circuit production line;
[0008] Enlarging or reducing the display screen peripheral circuit image to obtain an image to be tested whose size is consistent with the input size requirement of the defect detection model, wherein the defect detection model is obtained by training the fully convolutional network (FCN) algorithm using the historical defect display screen peripheral circuit image;
[0009] Inputting the image to be tested into a defect detection model to obtain a defect detection result;
[0010] The quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image is determined according to the defect detection result.
[0011] Optionally, in a possible implementation manner of the first aspect, before inputting the display screen peripheral circuit image into a defect detection model to obtain a defect detection result, the method further includes:
[0012] The FCN algorithm is trained on the defect detection model using the actual pixel category of the historical defect display screen peripheral circuit image, so that the loss value between the predicted pixel category output by the defect detection model for the historical defect display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold.
[0013] Optionally, in another possible implementation manner of the first aspect, before the display screen peripheral circuit image is enlarged or reduced, the method further includes:
[0014] Performing image preprocessing on the display screen peripheral circuit image, wherein the image preprocessing includes one or more of the following processes:
[0015] Crop, cut, rotate.
[0016] Optionally, in another possible implementation manner of the first aspect, inputting the image to be tested into a defect detection model to obtain a defect detection result includes:
[0017] Determine the detection model server that carries the processing resources according to the load balancing strategy;
[0018] The image to be tested is input into the defect detection model running on the detection model server to obtain a defect detection result.
[0019] Optionally, in another possible implementation of the first aspect, the defect detection result includes: defect category and / or defect pattern position;
[0020] Determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result includes:
[0021] The quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image is determined according to the production stage information and the defect detection result.
[0022] Optionally, in another possible implementation manner of the first aspect, after determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result, the method further includes:
[0023] If it is determined that the display screen peripheral circuit is a damaged circuit, one or more of the following operations are performed:
[0024] Send alarm information to production managers through the controller;
[0025] storing the defect detection results as logs in a production database through a controller;
[0026] Sending production control instructions to the console via the controller to eliminate defects;
[0027] The display screen peripheral circuit image and the defect detection result are input into the defect detection model to optimize the defect detection model.
[0028] A second aspect of the present invention provides a display screen peripheral circuit detection device, comprising:
[0029] A receiving module, configured to receive a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image acquired by an image acquisition device on the display screen peripheral circuit production line;
[0030] A preprocessing module, used to enlarge or reduce the display screen peripheral circuit image to obtain an image to be tested with a size consistent with the input size requirement of the defect detection model, wherein the defect detection model is obtained by training the fully convolutional network (FCN) algorithm using the historical defect display screen peripheral circuit image;
[0031] A processing module, used for inputting the image to be tested into a defect detection model to obtain a defect detection result;
[0032] A determination module is used to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result.
[0033] Optionally, in a possible implementation of the second aspect, the processing module is further used to perform the FCN algorithm training on the defect detection model with the actual pixel category of the historical defective display screen peripheral circuit image before inputting the display screen peripheral circuit image into the defect detection model to obtain the defect detection result, so that the loss value between the predicted pixel category output by the defect detection model for the historical defective display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold.
[0034] Optionally, in another possible implementation of the second aspect, the preprocessing module is further used to perform image preprocessing on the display screen peripheral circuit image before enlarging or reducing the display screen peripheral circuit image, wherein the image preprocessing includes one or more of the following processing: trimming, shearing, and rotation.
[0035] Optionally, in another possible implementation of the second aspect, the processing module is specifically used to determine the detection model server that carries processing resources according to the load balancing strategy; and input the image to be tested into the defect detection model running on the detection model server to obtain a defect detection result.
[0036] Optionally, in another possible implementation of the second aspect, the defect detection result includes: defect category and / or defect pattern position;
[0037] The determination module is specifically used to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the production stage information and the defect detection result.
[0038] Optionally, in another possible implementation manner of the second aspect, the processing module is further configured to, after determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result, if it is determined that the display screen peripheral circuit is a damaged circuit, perform one or more of the following operations:
[0039] Send alarm information to production managers through the controller;
[0040] storing the defect detection results as logs in a production database through a controller;
[0041] Sending production control instructions to the console via the controller to eliminate defects;
[0042] The display screen peripheral circuit image and the defect detection result are input into the defect detection model to optimize the defect detection model.
[0043] A third aspect of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method is implemented as described in the first aspect and any one of the various possible implementation methods of the first aspect.
[0044] A fourth aspect of the present invention provides a storage medium, wherein instructions are stored in the storage medium, and when the storage medium is run on a computer, the computer is caused to execute the method as described in the first aspect and any one of the various possible implementations of the first aspect.
[0045] The display screen peripheral circuit detection method, device, electronic device and storage medium provided by the present invention receive a quality detection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality detection request includes a display screen peripheral circuit image captured by an image acquisition device on the display screen peripheral circuit production line; the display screen peripheral circuit image is enlarged or reduced to obtain an image to be tested whose size is consistent with the input size requirement of a defect detection model, wherein the defect detection model is obtained by training a semantic segmentation algorithm FCN on a historical defect display screen peripheral circuit image; the image to be tested is input into the defect detection model to obtain a defect detection result; and the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image is determined according to the defect detection result. Since the above-mentioned defect detection model is obtained by training a historical defect display screen peripheral circuit image with FCN, the defect detection result obtained by using the defect detection model has high classification accuracy, strong intelligent capability, improved system performance, and high business scalability, which solves the problem of low detection accuracy, poor system performance, and low business scalability in the existing display screen peripheral circuit defect detection method due to the large subjective influence of human factors. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 A schematic diagram of the structure of a display screen peripheral circuit detection system provided by an embodiment of the present invention;
[0047] Figure 2 A schematic diagram of a flow chart of a first embodiment of a display screen peripheral circuit detection method provided by an embodiment of the present invention;
[0048] Figure 3 A schematic flow chart of a second embodiment of a display screen peripheral circuit detection method provided by an embodiment of the present invention;
[0049] Figure 4 A schematic diagram of the structure of an embodiment of a display screen peripheral circuit detection device provided by an embodiment of the present invention;
[0050] Figure 5 A schematic structural diagram of an electronic device embodiment provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0053] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.
[0054] It should be understood that in the present invention, "multiple" means two or more. "And / or" is just a way to describe the association relationship of associated objects, indicating that there can be three relationships. For example, and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0055] It should be understood that in the present invention, "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based only on A, but B can also be determined based on A and / or other information. A and B match when the similarity between A and B is greater than or equal to a preset threshold.
[0056] Depending on the context, "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting."
[0057] At present, the overall intelligent automation level of the 3C industry (3C industry refers to the information appliance industry that integrates the three major technological products of computers, communications, and consumer electronics) is relatively low. Through research and analysis of the peripheral circuit industry of mobile phone screens and other display screens, it can be seen that most manufacturers use two types of detection methods for mobile phone screens, namely: manual detection method and machine-assisted manual detection method.
[0058] Among them, the manual detection method refers to relying on industry experts to observe the images collected from the production environment for judgment. This method is greatly affected by human subjective factors, has low detection efficiency, and is more harmful to the human eye. In addition, since the display peripheral circuit production workshop is generally a dust-free environment, workers need to clean and prepare before entering and wear dust-free clothes, which may also have an adverse effect on the health and safety of workers.
[0059] The machine-assisted manual inspection method can also be called the inspection method based on the LCD module inspection equipment. The specific principle is: first, the quality inspection system with certain judgment ability filters out the images without defects, and then the industry experts detect and judge the images suspected of having defects. In the machine-assisted manual inspection method, the quality inspection system is mostly developed from the expert system and feature engineering system, which means that the experts solidify their experience in the quality inspection system to give it a certain degree of automation. Therefore, the machine-assisted manual inspection method not only has low accuracy and poor system performance, and cannot cover all the inspection standards of the manufacturer, but also has low efficiency, is prone to missed judgments and misjudgments, and the image data after inspection is difficult to mine for secondary use. In addition, in the above-mentioned quality inspection system, the features and judgment rules are solidified into the machine based on the experience of industry experts, and it is difficult to iterate with the development of the business, resulting in the quality inspection system The detection accuracy is getting lower and lower with the development of production technology, and may even be reduced to a completely unusable state. In addition, the features of the quality inspection system are pre-solidified in the hardware by third-party suppliers. Upgrading not only requires major changes to the production line, but is also expensive. It has obvious deficiencies in terms of security, standardization, and scalability, which is not conducive to the optimization and upgrade of the display peripheral circuit production line and has low business expansion capabilities.
[0060] In summary, both manual inspection methods and machine-assisted manual inspection methods have the following disadvantages: not only are they inefficient and prone to misjudgment, but the industrial data generated by these two methods are also difficult to store, manage, and reuse for secondary mining.
[0061] The embodiment of the present invention is based on the latest development of artificial intelligence technology in computer vision, and develops an automated, high-precision, adaptively corrected and upgraded display screen peripheral circuit detection method. The display screen peripheral circuit image captured in real time on the display screen peripheral circuit production line by an image acquisition device is used to detect and judge the surface quality of the display screen peripheral circuit in real time. If it is detected that the display screen peripheral circuit captured by the current image acquisition device has defects, the position of the defect in the image and the category of the defect are determined.
[0062] Optionally, the defects described in the embodiments of the present invention may include, but are not limited to, different types of defect problems including point defects, foreign body defects, and scratch defects, etc. They will not be introduced one by one here.
[0063] It should be understood that in the present invention, the fully convolutional network (FCN) algorithm converts the fully connected layer in the traditional convolutional neural network (CNN) into a convolutional layer. Compared with the classic CNN network, FCN converts the last three fully connected layers into three convolutional layers. In the classic CNN structure, the first 5 layers are convolutional layers, the 6th and 7th layers are respectively a one-dimensional vector with a length of 4096, and the 8th layer is a one-dimensional vector with a length of 1000, corresponding to the probabilities of 1000 different categories. FCN converts these 3 layers into convolutional layers, and the sizes of the convolution kernels (number of channels, width, height) are (4096, 1, 1), (4096, 1, 1), (1000, 1, 1) respectively. It seems that there is no difference in numbers, but convolution and full connection are different concepts and calculation processes. The weights and biases that have been trained by CNN before are used, but the difference is that the weights and biases have their own ranges and belong to their own convolution kernels. Therefore, all layers in the FCN network are convolutional layers, so it is called a fully convolutional network.
[0064] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0065] The following first briefly describes the application scenarios to which the embodiments of the present invention are applicable. Figure 1 , is a schematic diagram of the structure of a display screen peripheral circuit detection system provided by an embodiment of the present invention. Figure 1 In the system shown, the display screen peripheral circuit detection method provided by the present invention is applied to detect defects in the display screen peripheral circuit. Figure 1 As shown, the display screen peripheral circuit detection system mainly includes: a control console 12, a server group 13, a controller 14, a database 15, a trainer 16 and an image acquisition device 11 deployed on the display screen peripheral circuit production line.
[0066] Among them, the image acquisition device 11 acquires the display screen peripheral circuit image on the display screen peripheral circuit production line, the control console 12 receives the display screen peripheral circuit image acquired by the image acquisition device 11, and sends the display screen peripheral circuit image to the detection model server 130 in the server group 13, the detection model server 130 inputs the received display screen peripheral circuit image into the defect detection model running by itself to obtain the defect detection result, the controller 14 receives the defect detection result of the detection model server 130, and gives a business response in combination with the production stage information, and the controller 14 can also store the defect detection result as a log in the database 15. In addition, the display screen peripheral circuit image acquired by the image acquisition device 11 can also be directly stored in the database 15 as the original data for defect detection model training. The trainer 16 extracts the historical defect display screen peripheral circuit image in the database and trains the defect detection model based on the Fully Convolutional Network (FCN) algorithm.
[0067] Optionally, the above-mentioned database 15 may include a production database 151 and a training database 152. The production database 151 can receive and save the defect detection results sent by the controller 14 and the display screen peripheral circuit images captured by the image acquisition device 11. The training database 152 can store the historical defect display screen peripheral circuit images extracted from the production database 151 and the corresponding original display screen peripheral circuit images, so that the trainer 16 can train a defect detection model with high detection accuracy.
[0068] Optionally, the trainer 16 in the embodiment of the present invention may be a training engine implemented by hardware and / or software functions, as a training tool for the defect detection model. The display screen peripheral circuit detection system in the embodiment of the present invention may also include other entity modules such as a processor and a memory, but the present embodiment is not limited thereto.
[0069] See also Figure 2 , is a flow chart of a first embodiment of a display screen peripheral circuit detection method provided by an embodiment of the present invention, Figure 2 The execution subject of the method shown can be a software device, a hardware device, or a device combining software and hardware. It includes steps S101 to S104, which are as follows:
[0070] S101, receiving a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image captured by an image capture device on the display screen peripheral circuit production line.
[0071] Optionally, in an embodiment of the present invention, a plurality of different devices such as image acquisition devices, consoles, server groups, controllers, and databases are deployed on the display peripheral circuit production line. The image acquisition device may be a high-precision image acquisition camera. During the production process of the display peripheral circuit, multiple display peripheral circuit images corresponding to the display peripheral circuit in the production process may be acquired by adjusting the angle, light, filter, magnification, and focus of the image acquisition device.
[0072] After the image acquisition device on the display screen peripheral circuit production line captures the display screen peripheral circuit image, the control console deployed on the display screen peripheral circuit production line can send a quality inspection request to the server group on the display screen peripheral circuit production line that has a defect detection model deployed thereon. The quality inspection request includes the display screen peripheral circuit image captured by the above-mentioned image acquisition device, so that the server in the server group that receives the quality inspection request processes the received display screen peripheral circuit image.
[0073] S102, enlarging or reducing the display screen peripheral circuit image to obtain an image to be tested whose size is consistent with the input size requirement of the defect detection model.
[0074] S103, inputting the image to be tested into a defect detection model to obtain a defect detection result.
[0075] Among them, the defect detection model is obtained by training the fully convolutional network FCN algorithm using the peripheral circuit images of the historical defective display screen. The defect detection model trained by the FCN algorithm has size requirements for the input image. Once the size of the input image is inconsistent with the size required by the model input, the defect detection model will not be able to process it. When testing the peripheral circuit of the display screen, the line direction and winding shape indicated by the global image can better reflect the possible defects. Therefore, in this embodiment, before inputting the defect detection model, the peripheral circuit graphics of the display screen are first scaled to make the size of the image to be tested consistent with the input size requirement of the defect detection model.
[0076] Zooming in or out of the display screen peripheral circuit image can be understood as zooming in or out with unchanged pixels, or zooming in or out with reduced pixels. Too high a pixel may exceed the processing capability of the defect detection model, so when the display screen peripheral circuit image has too high a pixel, the display screen peripheral circuit image may also be firstly processed to reduce the pixel, which is not limited here.
[0077] Optionally, the server receiving the quality inspection request obtains the display screen peripheral circuit image in the quality inspection request, and performs preprocessing of enlarging or reducing it to obtain an image to be tested whose size is consistent with the input size requirement of the defect detection model. The image to be tested is then input into the defect detection model running on the server, and the defect detection model performs defect detection to obtain a defect detection result.
[0078] In one implementation, before the display screen peripheral circuit image is enlarged or reduced, the display screen peripheral circuit image may be preprocessed, wherein the image preprocessing includes one or more of the following processes: trimming, shearing, and rotation. It can be understood that the image acquisition device deployed on the display screen peripheral circuit production line is generally a high-precision camera, and therefore, the display screen peripheral circuit image acquired by the image acquisition device may be large in size, or have high pixels, or be inappropriately positioned, etc. Therefore, after receiving the display screen peripheral circuit image included in the quality inspection request sent by the control console, the display screen peripheral circuit image needs to be preprocessed according to the actual situation. For example, if the edge area of the display screen peripheral circuit image is large, at this time, the display screen peripheral circuit image can be trimmed to retain the useful part of the display screen peripheral circuit image.
[0079] It is worth noting that the defect detection model running on the server is obtained by training the full convolution network FCN algorithm on the peripheral circuit images of the historical defective display screen. Specifically, the present embodiment uses the FCN algorithm to perform image semantic segmentation. Image semantic segmentation refers to allowing the computer to perform segmentation based on the semantics of the image. Semantics in the image field refers to the content of the image, and segmentation means segmenting different types of objects in the image from the perspective of pixels, and each pixel in the original image is labeled with a type. For example, each pixel is labeled as a normal pixel, a first-class defective pixel, or a second-class defective pixel. Therefore, in the embodiment of the present invention, the defect detection model is obtained by performing image semantic segmentation training on a large number of historical defective display screen peripheral circuit images. Specifically, both normal pixels and defective pixels in the historical defective display screen peripheral circuit images are labeled, and the labeling of defective pixels also distinguishes their defect types. Then, the historical defective display screen peripheral circuit images with each pixel labeled are used as training sets and test sets to perform FCN algorithm recognition, and the loss function is obtained according to the output of the training model and the labeled pixel type. The loss function is used to back-propagate the model, train and learn the model until the loss value corresponding to the loss function is lower than the preset loss threshold, and the training is completed, thereby obtaining a defect detection model.
[0080] In the embodiment of the present invention, the defect detection model adopts the FCN structure. Specifically, the display screen peripheral circuit image on the display screen peripheral circuit production line is used as the input of the defect detection model, and the FCN structure of the defect detection model is used to identify the characteristics of each pixel in the display screen peripheral circuit image, that is, to obtain which pixels in the display screen peripheral circuit image are normal pixels and which pixels are defective pixels.
[0081] As an example, before the display screen peripheral circuit image is input into the defect detection model to obtain the defect detection result, a model training process may also be included. Specifically, the defect detection model may be trained with the FCN algorithm using the actual pixel category of the historical defect display screen peripheral circuit image, so that the loss value between the predicted pixel category output by the defect detection model for the historical defect display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold.
[0082] In the FCN model, all layers are implemented by convolution, which can be understood as all layers are convolution layers. The embodiment of the present invention can use this model with convolution, pooling, deconvolution upsampling operations, which has high robustness to the deformation, blur, illumination change and other features of the display screen peripheral circuit image collected by the image acquisition device on the display screen peripheral circuit production line, and has higher generalizability for classification tasks.
[0083] It is worth noting that, in the embodiment of the present invention, for different production scenarios and characteristics of the display screen peripheral circuit image, the organization method of the FCN model required for training the above-mentioned defect detection model may be different, which can be determined according to actual conditions, and this embodiment does not limit it.
[0084] S104, determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result.
[0085] Optionally, in an embodiment of the present invention, after a defect detection result is obtained according to the defect detection model, the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image may be determined based on the defect detection result.
[0086] Optionally, in one embodiment of the present invention, the above-mentioned defect detection result may include: defect category and / or defect pattern location. For example, when there are defects in the peripheral circuit image of the display screen, the defect detection result obtained by the defect detection model may include defect category (how many types of defects exist in the peripheral circuit of the display screen) and defect location (the specific location of each defect). The presentation of the defect detection result can be understood as the defect detection model outputs a segmentation map, and the segmentation map uses the first color to identify normal pixels, the second color to identify the first type of defects, and the second color to identify the second type of defects. In the defect detection result in which two types of defects are detected, for example, it can be a segmentation map with a white background and containing blue blocks and green blocks, where white represents pixels in the normal area, blue represents pixels in the point defect area, and green represents pixels in the foreign body defect area. The FCN model is pixel recognition, so various defect patterns can be obtained from the defect detection results, which can be understood as the contour shape of various defects and their pixel positions in the peripheral circuit image of the display screen.
[0087] Correspondingly, S104 (determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image based on the defect detection result) can be replaced by: determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image based on the production stage information and the defect detection result.
[0088] Specifically, various production stage information such as the manufacturer, production environment, and type of the display peripheral circuit may result in different defect detection results during the display peripheral circuit detection process. Different types of display peripheral circuits have gone through different production stages. Therefore, when analyzing the defect detection results obtained above, it is necessary to combine the production stage information of each display peripheral circuit to determine the quality of the display peripheral circuit.
[0089] It is worth noting that the defect detection model of the embodiment of the present invention can detect several types of defects in the peripheral circuit image of the display screen. The embodiment of the present invention does not limit the specific number of each type of defects, that is, the defect detection model obtained by using the FCN algorithm does not distinguish between different defect individuals belonging to the same category.
[0090] The display screen peripheral circuit detection method provided by the embodiment of the present invention receives a quality detection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality detection request includes a display screen peripheral circuit image acquired by an image acquisition device on the display screen peripheral circuit production line, inputs the display screen peripheral circuit image into a defect detection model to obtain a defect detection result, and determines the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result. Since the above-mentioned defect detection model is obtained by training the historical defect display screen peripheral circuit image with an FCN algorithm, the defect detection result obtained by using the defect detection model has high classification accuracy, strong intelligent capability, improved system performance, and high business scalability.
[0091] See also Figure 3 , is a flow chart of Embodiment 2 of the display screen peripheral circuit detection method provided by the embodiment of the present invention. Based on the above embodiment, Figure 3 In the illustrated embodiment, the above S104 (inputting the image to be tested into the defect detection model to obtain a defect detection result) can be implemented by steps S301-S302, which are specifically as follows:
[0092] S301, determining a detection model server that carries processing resources according to a load balancing strategy.
[0093] Optionally, in an embodiment of the present invention, a server group is deployed on the display peripheral circuit production line, and the number of servers in the server group can be multiple, and a defect detection model is running on each server. Optionally, the defect detection model running on each server is the same, so each server can receive the quality detection request sent by the console, and then can use the defect detection model carried by itself to perform quality detection on the display peripheral circuit image.
[0094] As an example, since the image acquisition device deployed on the display screen peripheral circuit production line acquires the display screen peripheral circuit image in real time, the console can also send a quality inspection request to any server in the server group in real time.
[0095] Optionally, since the defect detection model running on each server in the server group is the same, in order to improve the detection efficiency of the defect detection model on the server and ensure the load balancing of the defect detection model, a server can be determined from the server group as the detection model server that carries the processing resources according to a pre-set load balancing strategy, that is, load balancing and scheduling can be performed in real time according to the deployment of the defect detection model on the display peripheral circuit production line.
[0096] S302: Input the image to be tested into the defect detection model running on the detection model server to obtain a defect detection result.
[0097] Optionally, in an embodiment of the present invention, after determining the detection model server that carries the processing resources from the server group, the above-mentioned display screen peripheral circuit image can be input into the defect detection model running on the detection model server, and the defect detection model is used to detect the defects of the display screen peripheral circuit image, thereby obtaining a defect detection result. Optionally, the defect detection model is obtained by training the preset pixel categories and actual pixel categories in the historical defect display screen peripheral circuit image by the training module.
[0098] The display screen peripheral circuit detection method provided in the embodiment of the present invention determines the detection model server that carries the processing resources according to the load balancing strategy, and inputs the above-mentioned image to be tested into the defect detection model running on the above-mentioned detection model server to obtain the defect detection result. It can achieve load balancing on the server, improve the detection efficiency of the display screen peripheral circuit image, and enhance the performance of the display screen peripheral circuit detection system.
[0099] In one implementation, after the above step S302 (inputting the image to be tested into the defect detection model running on the detection model server to obtain a defect detection result), the following steps may also be included:
[0100] If it is determined that the display screen peripheral circuit is a damaged circuit, one or more of the following operations are performed:
[0101] Send alarm information to production managers through the controller;
[0102] storing the defect detection results as logs in a production database through a controller;
[0103] Sending production control instructions to the console via the controller to eliminate defects;
[0104] The display screen peripheral circuit image and the defect detection result are input into the defect detection model to optimize the defect detection model.
[0105] Optionally, in an embodiment of the present invention, the tester can pre-set a solution when the peripheral circuit of the display screen is determined to be a bad screen based on the production scenario and production stage information of the peripheral circuit of the display screen, such as sending an alarm message to the production manager through the controller, and / or storing the above defect detection results as logs in the production database through the controller, and / or sending production control instructions to the console through the controller to eliminate defects, and / or inputting the above display screen peripheral circuit image and the above defect detection results into the above defect detection model to optimize the above defect detection model, etc.
[0106] Specifically, as an example, when it is determined according to the above defect detection results that the display screen peripheral circuit corresponding to the display screen peripheral circuit image is a damaged circuit, that is, there is a defect in the display screen peripheral circuit, an alarm message can be issued to enable the production manager to locate the type and position of the defect in time and provide a solution.
[0107] As another example, when it is determined that there is a defect in the peripheral circuit of the display screen based on the above defect detection results, the above defect detection results can be stored as logs in the production database through the controller, that is, the defect category of the peripheral circuit of the display screen and / or the defect graphic position are stored as logs in the production database, and then they can be filtered into the training database, and the training module (which can be a software program such as a training engine) updates the above defect detection model according to the image of the defective peripheral circuit of the display screen.
[0108] As another example, when it is determined that there is a defect in the peripheral circuit of the display screen according to the above defect detection results, a production control instruction can be sent to the console through the controller to eliminate the defect. That is, the detection model server carrying the defect detection model can determine the cause of the defect through the controller, and then adjust the production process accordingly, that is, the detection model server sends a production control instruction to the console through the controller to eliminate the defect in the peripheral circuit of the display screen, so as to reduce the probability of damaged circuits.
[0109] As another example, when it is determined that there is a defect in the peripheral circuit of the display screen based on the above defect detection results, the above display screen peripheral circuit image and the above defect detection results can also be directly input into the above defect detection model to optimize the above defect detection model, that is, the display screen peripheral circuit image corresponding to the damaged circuit is directly used as a training set of the defect detection model to optimize the defect detection model, thereby improving the detection accuracy of the defect detection model.
[0110] It is worth noting that the embodiment of the present invention does not limit the above one or more operations that can be performed by the detection model server when determining that the peripheral circuit of the display screen is a damaged circuit. It can be determined according to actual conditions and will not be repeated here.
[0111] Optionally, for multiple different devices deployed on the display peripheral circuit production line, such as image acquisition devices, consoles, server groups, controllers, and databases, the operation steps corresponding to the display peripheral circuit detection method can also be distributed to the above multiple different devices for execution. For example, the image acquisition device acquires the display peripheral circuit image, and the console sends the display peripheral circuit image acquired by the image acquisition device to the detection model server in the server group according to the load balancing strategy. The defect detection model running on the detection model server performs a preset preprocessing on the display peripheral circuit image and then performs defect detection, and gives the defect detection result. The detection model server can send the defect detection result to the controller. On the one hand, the controller combines the actual business scenario and responds to the above defect detection result according to the business needs, such as alarm, storage log, control production control instructions, etc. On the other hand, the controller can also store the defect detection result and the above response processing behavior as a log in the production database, so that the training module updates the above defect detection model according to the display peripheral circuit image and defect detection result in the training database. The training database stores the defective display peripheral circuit image and the corresponding defect detection result screened from the production database.
[0112] It is worth noting that for each optimized defect detection model, the defect detection model currently running on the server can be gradually replaced by a small amount of traffic online, so as to achieve the purpose of dynamic expansion and generalization of the defect detection model with business scenarios and production stage information. After the display peripheral circuit detection method in the embodiment of the present invention has been running on the display peripheral circuit production line for a period of time, the accuracy of the above-mentioned defect detection and defect location can be manually reviewed through the information in the production database, and then the above-mentioned training database can be updated to retrain the defect detection model to improve the accuracy of defect detection.
[0113] The following are embodiments of the device of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.
[0114] See also Figure 4 , is a schematic diagram of the structure of an embodiment of a display screen peripheral circuit detection device provided by an embodiment of the present invention. Figure 4 As shown, the display screen peripheral circuit detection device provided by the embodiment of the present invention may mainly include: a receiving module 41 , a pre-processing module 42 , a processing module 43 and a determining module 44 .
[0115] The receiving module 41 is used to receive a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image captured by an image capture device on the display screen peripheral circuit production line.
[0116] The preprocessing module 42 is used to enlarge or reduce the peripheral circuit image of the display screen to obtain an image to be tested whose size is consistent with the input size requirement of the defect detection model, wherein the defect detection model is obtained by training the fully convolutional network FCN algorithm using the historical defective display screen peripheral circuit image.
[0117] The processing module 43 is used to input the image to be tested into the defect detection model to obtain the defect detection result.
[0118] The determination module 44 is used to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result.
[0119] Figure 4 The display screen peripheral circuit detection device of the embodiment shown can be used to perform Figure 2 The implementation principles and technical effects of the steps in the method embodiment shown are similar and will not be repeated here.
[0120] Optionally, the processing module 43 is further used to perform the FCN algorithm training on the defect detection model with the actual pixel category of the historical defective display screen peripheral circuit image before inputting the display screen peripheral circuit image into the defect detection model to obtain the defect detection result, so that the loss value between the predicted pixel category output by the defect detection model for the historical defective display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold.
[0121] Optionally, the preprocessing module 42 is further used to perform image preprocessing on the display screen peripheral circuit image before enlarging or reducing the display screen peripheral circuit image, wherein the image preprocessing includes one or more of the following processes: trimming, shearing, and rotation.
[0122] Optionally, the processing module 43 is specifically used to determine the detection model server that carries the processing resources according to the load balancing strategy; input the image to be tested into the defect detection model running on the detection model server to obtain the defect detection result.
[0123] Optionally, the defect detection result includes: defect category and / or defect pattern position.
[0124] The determination module 44 is specifically configured to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the production stage information and the defect detection result.
[0125] Optionally, the processing module 43 is further configured to, after determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result, if it is determined that the display screen peripheral circuit is a damaged circuit, perform one or more of the following operations:
[0126] Send alarm information to production managers through the controller;
[0127] storing the defect detection results as logs in a production database through a controller;
[0128] Sending production control instructions to the console via the controller to eliminate defects;
[0129] The display screen peripheral circuit image and the defect detection result are input into the defect detection model to optimize the defect detection model.
[0130] The display screen peripheral circuit detection device of the above device embodiment can be used to perform Figures 2 to 3 The implementation scheme of the method embodiment shown in the figure is similar in specific implementation method and technical effect, which will not be repeated here.
[0131] See also Figure 5 , is a schematic diagram of the structure of an electronic device embodiment provided by an embodiment of the present invention, the electronic device includes: a processor 51, a memory 52 and a computer program; wherein
[0132] The memory 52 is used to store the computer program, and the memory may also be a flash memory. The computer program is, for example, an application program, a functional module, etc. for implementing the above method.
[0133] The processor 51 is used to execute the computer program stored in the memory to implement the various steps performed by the electronic device in the above method. For details, please refer to the relevant description in the above method embodiment.
[0134] Optionally, the memory 52 may be independent or integrated with the processor 51 .
[0135] When the memory 52 is a device independent of the processor 51, the electronic device may further include:
[0136] The bus 53 is used to connect the memory 52 and the processor 51 .
[0137] The present invention also provides a storage medium, wherein instructions are stored in the storage medium, and when the instructions are run on a computer, the computer is caused to execute the following Figures 2 to 3 The method of the method embodiment is shown.
[0138] Among them, the storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, the storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (Application Specific Integrated Circuits, abbreviated as: ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also exist in a communication device as discrete components.
[0139] The present invention also provides a program product, which includes a computer program stored in a storage medium. At least one processor of the display screen peripheral circuit detection device can read the computer program from the storage medium, and at least one processor executes the computer program so that the display screen peripheral circuit detection device performs Figures 2 to 3 The method of the method embodiment is shown.
[0140] In the above-mentioned embodiment of the electronic device, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.
[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A display screen peripheral circuit detection method, characterized in that: include: Receiving a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image acquired by an image acquisition device on the display screen peripheral circuit production line; The display screen peripheral circuit image is enlarged or reduced to obtain a test image whose size is consistent with the input size requirement of the defect detection model, wherein the defect detection model is obtained by training the full convolution network FCN algorithm using the actual pixel category of the historical defect display screen peripheral circuit image, and the loss value between the predicted pixel category output by the defect detection model for the historical defect display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold; wherein the FCN algorithm is used for image semantic segmentation, and the image semantic segmentation refers to segmenting different types of objects in the picture from the perspective of pixels according to the semantics of the image, and marking each pixel as a normal pixel or a defective pixel, and the defective pixel also distinguishes different defect types; Inputting the image to be tested into a defect detection model to obtain a defect detection result, wherein the defect detection model outputs a segmentation map, wherein the segmentation map identifies normal pixels and different defective pixels with different colors; Determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result; The defect detection result includes: defect category and / or defect pattern location; Determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result includes: Determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the production stage information and the defect detection result, wherein different types of display screen peripheral circuits have different production stage information, and the production stage information includes the manufacturer, production environment and type of the display screen peripheral circuit; If it is determined that the display screen peripheral circuit is a damaged circuit, the method further includes: Sending a production control instruction to the console through a controller, wherein the production control instruction is used to adjust the production process based on the cause of the defect to eliminate the defect; The display screen peripheral circuit image and the defect detection result are input into the defect detection model, so as to gradually replace the defect detection model running on the server by adopting a small traffic online method.
2. The method according to claim 1, characterized in that Before the image of the peripheral circuit of the display screen is enlarged or reduced, the method further includes: Performing image preprocessing on the display screen peripheral circuit image, wherein the image preprocessing includes one or more of the following processes: Crop, cut, rotate.
3. The method according to claim 1 or 2, characterized in that: The step of inputting the image to be tested into a defect detection model to obtain a defect detection result includes: Determine the detection model server that carries the processing resources according to the load balancing strategy; The image to be tested is input into the defect detection model running on the detection model server to obtain a defect detection result.
4. The method according to any one of claims 1 to 3, characterized in that: After determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result, the method further includes: If it is determined that the display screen peripheral circuit is a damaged circuit, one or more of the following operations are performed: Send alarm information to production managers through the controller; The defect detection results are stored as logs in a production database through a controller.
5. A display screen peripheral circuit detection device, characterized in that: include: A receiving module, configured to receive a quality inspection request sent by a control console deployed on a display screen peripheral circuit production line, wherein the quality inspection request includes a display screen peripheral circuit image acquired by an image acquisition device on the display screen peripheral circuit production line; A preprocessing module, used for enlarging or reducing the display screen peripheral circuit image to obtain a test image whose size is consistent with the input size requirement of the defect detection model, wherein the defect detection model is obtained by training the full convolution network FCN algorithm with the actual pixel category of the historical defect display screen peripheral circuit image, and the loss value between the predicted pixel category output by the defect detection model for the historical defect display screen peripheral circuit image and the actual pixel category is lower than a preset loss threshold; wherein the FCN algorithm is used for image semantic segmentation, and the image semantic segmentation refers to segmenting different types of objects in the picture from the perspective of pixels according to the semantics of the image, and marking each pixel as a normal pixel or a defective pixel, and the defective pixel also distinguishes different defect types; A processing module, used for inputting the image to be tested into a defect detection model to obtain a defect detection result, wherein the defect detection model outputs a segmentation map, and the segmentation map identifies normal pixels and different defective pixels with different colors; A determination module, used to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result; The defect detection result includes: defect category and / or defect pattern location; The determination module is specifically used to determine the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the production stage information and the defect detection result. Different types of display screen peripheral circuits have different production stage information. The production stage information includes the manufacturer, production environment and type of the display screen peripheral circuit. If it is determined that the peripheral circuit of the display screen is a damaged circuit, the processing module is also used to send a production control instruction to the console through the controller, and the production control instruction is used to adjust the production process based on the cause of the defect to eliminate the defect, and input the display screen peripheral circuit image and the defect detection result into the defect detection model, so as to gradually replace the defect detection model running on the server by using a small traffic online method.
6. The device according to claim 5, characterized in that The preprocessing module is further used to perform image preprocessing on the display screen peripheral circuit image before enlarging or reducing the display screen peripheral circuit image, wherein the image preprocessing includes one or more of the following processes: trimming, shearing, and rotation.
7. The device according to claim 5 or 6, characterized in that The processing module is specifically used to determine the detection model server that carries the processing resources according to the load balancing strategy; input the image to be tested into the defect detection model running on the detection model server to obtain the defect detection result.
8. The device according to claim 5 or 6, characterized in that The processing module is further configured to, after determining the quality of the display screen peripheral circuit corresponding to the display screen peripheral circuit image according to the defect detection result, if it is determined that the display screen peripheral circuit is a damaged circuit, perform one or more of the following operations: Send alarm information to production managers through the controller; The defect detection results are stored as logs in a production database through a controller.
9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.
10. A storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Technology for preparing high precision ultrathin glass substrate
CN101456668A
Defect detection and response
CN101960579A
Production technology for hot-continuous-rolled strip steel
CN103191918A
Automatic detection method of massively produced bands
CN103674970A
Method for automatically detecting defects of wheel-shaped cast product
CN105118044A
