Data collaboration method and system based on multi-modal detection data self-detection camera

Abstract

The application relates to the technical field of computer vision, and relates to a data cooperation method and system for self-detecting a camera based on multi-modal detection data, which comprises the following steps: collecting appearance images of a marking panel to obtain the appearance images, acquiring appearance confidence scores, confirming a plurality of appearance qualified panels and a plurality of appearance abnormal panels, detecting display states of the plurality of appearance qualified panels to obtain a plurality of display confidence scores, confirming a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores, collecting infrared thermal imaging of the display abnormal panels to obtain thermal distribution images, acquiring abnormal circuit coordinate sets, and collecting a plurality of abnormal circuit coordinate sets corresponding to the plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels and the plurality of display abnormal panels to obtain a detection report. The application can solve the problems of low effective defect information extraction efficiency in OLED panel detection and server resource waste caused by full-quantity uploading data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a data collaboration method and system for self-detecting a camera based on multi-modal detection data. BACKGROUND

[0002] With the rapid development of OLED panel production, the quality detection of OLED panels faces new challenges. By performing multi-modal detection analysis at the front end, key defect data can be confirmed according to defect confidence and uploaded to a server, thereby realizing efficient and accurate data collaboration to solve the problems of low effective defect information extraction efficiency and server resource waste caused by full-quantity uploaded data in OLED panel detection.

[0003] Currently, the detection data collected is mainly uploaded to the server for subsequent quality screening. Although the traditional data collaboration method can realize the transmission of detection data, there is a problem that the server resources are occupied by a large amount of invalid data, and efficient and accurate detection cannot be realized. Therefore, it is of great significance to optimize the data collaboration method of multi-modal detection data self-detection camera for improving the detection efficiency of OLED panels. SUMMARY

[0004] The present application provides a data collaboration method for self-detecting a camera based on multi-modal detection data and a computer readable storage medium, which mainly aims to solve the problems of low effective defect information extraction efficiency and server resource waste caused by full-quantity uploaded data in OLED panel detection.

[0005] To achieve the above-mentioned purpose, the present application provides a data collaboration method for self-detecting a camera based on multi-modal detection data, which comprises:

[0006] Classifying and marking a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels;

[0007] For each marked panel in the plurality of marked panels, the following operations are performed:

[0008] Collecting appearance images of the marked panel to obtain appearance images;

[0009] Obtaining appearance confidence scores based on the appearance images;

[0010] Summarizing the appearance confidence scores to obtain a plurality of appearance confidence scores;

[0011] Confirming a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores;

[0012] Performing display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores;

[0013] Based on the plurality of display confidence scores, a plurality of display qualified panels and a plurality of display abnormal panels are confirmed;

[0014] For each of the plurality of display abnormal panels, the following operations are performed:

[0015] Infrared thermal imaging collection is performed on the display abnormal panel to obtain a thermal distribution image;

[0016] Based on the thermal distribution image, an abnormal circuit coordinate set is obtained, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates;

[0017] The abnormal circuit coordinate set is summarized to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to one display abnormal panel;

[0018] The plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels, and the plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels are summarized to obtain a detection report.

[0019] Optionally, the pre-confirmed plurality of OLED panels are classified and labeled to obtain a plurality of labeled panels, including:

[0020] Based on the plurality of OLED panels, a plurality of panel images are obtained, wherein the OLED panel and the panel image correspond one-to-one;

[0021] For each of the plurality of panel images, the following operations are performed:

[0022] Contour extraction is performed on the panel image to obtain a contour image;

[0023] Based on a pre-constructed panel model template library, the contour image is classified and recognized to obtain a panel model;

[0024] The panel models are summarized to obtain a plurality of panel models;

[0025] Based on the plurality of panel models, a plurality of labeled panels are obtained.

[0026] Optionally, appearance image collection is performed on the labeled panel to obtain an appearance image, including:

[0027] Based on the labeled panel, a detection area definition parameter is obtained;

[0028] Image collection is performed on the labeled panel to obtain a labeled panel image;

[0029] Based on the detection area definition parameter, the labeled panel image is classified and extracted to obtain a special-shaped structure image;

[0030] Obtain the target panel image using the marked panel image and the image of the irregular structure;

[0031] The target panel image is preprocessed to obtain the appearance image.

[0032] Optionally, obtaining the appearance confidence score based on the appearance image includes:

[0033] A full-area scan of the appearance image is performed to obtain a grayscale distribution image;

[0034] The defect pixel set is identified using the grayscale distribution image and the pre-confirmed standard grayscale image, wherein one or more defect pixels exist in the defect pixel set;

[0035] An initial defect location map is constructed based on the defect pixel set;

[0036] Using the initial defect location map and the pre-built defect screening model, multiple defect types and multiple defect areas were identified, wherein the defect type and the defect area corresponded one-to-one.

[0037] The appearance confidence score is calculated based on multiple defect types and multiple defect areas.

[0038] Optionally, the calculation of the appearance confidence score based on multiple defect types and multiple defect areas includes:

[0039] Multiple defect weights are obtained by using multiple defect types and a pre-built defect type-weight mapping table, where each defect weight corresponds one-to-one with a defect type;

[0040] Obtain the number of defect types and the area of ​​the detection region;

[0041] The appearance confidence score is calculated using multiple defect weights, multiple defect areas, the number of defect types, and the area of ​​the inspection region. The formula for calculating the appearance confidence score is as follows:

[0042] ,

[0043] in, This represents the appearance confidence score. Indicates the number of defect types. Represents the weight of the first defect among multiple defect weights. Each defect weight Represents the area of ​​multiple defects. Area of ​​each defect Indicates the area of ​​the detection region. This indicates taking the maximum value.

[0044] Optionally, the step of performing display status detection on the plurality of appearance-qualified panels to obtain multiple display confidence scores includes:

[0045] performing the following operations on each of the plurality of appearance qualified panels:

[0046] obtaining four standard pure color pictures;

[0047] performing the following operations on each of the four standard pure color pictures:

[0048] obtaining a lighted panel based on the appearance qualified panel and the standard pure color pictures;

[0049] performing image acquisition on the lighted panel to obtain a display image;

[0050] performing effective area extraction on the display image to obtain an effective display area image;

[0051] performing defect quantification processing on the effective display area image to obtain a channel display score;

[0052] summarizing the channel display scores to obtain a plurality of channel display scores;

[0053] calculating a display confidence score based on the plurality of channel display scores;

[0054] summarizing the display confidence scores to obtain a plurality of display confidence scores.

[0055] Optionally, the defect quantification processing on the effective display area image to obtain a channel display score comprises:

[0056] performing brightness statistics on the effective display area image to obtain an average brightness and a brightness standard deviation;

[0057] obtaining a brightness offset degree using the average brightness and a pre-confirmed standard brightness;

[0058] calculating the channel display score using the brightness offset degree and the brightness standard deviation, wherein the calculation formula of the channel display score is as follows:

[0059] ,

[0060] wherein, represents the channel display score, represents a pre-set brightness deviation coefficient, represents the brightness offset degree, represents a pre-set maximum brightness offset degree, represents a pre-set uniformity index coefficient, represents the brightness standard deviation, represents a pre-set maximum brightness standard deviation, represents taking the minimum value.

[0061] Optionally, the infrared thermal imaging acquisition is performed on the display abnormal panel to obtain a thermal distribution image, including:

[0062] The infrared acquisition parameter is acquired based on the display abnormal panel;

[0063] The infrared image acquisition is performed on the display abnormal panel according to the preset acquisition time interval by using the infrared acquisition parameter, and a thermal image sequence is obtained;

[0064] The thermal image sequence is superimposed and filtered and denoised to obtain a denoised image;

[0065] The denoised image is calibrated by using the pre-confirmed black body reference data to obtain an apparent temperature distribution map;

[0066] The emissivity of the display abnormal panel is acquired;

[0067] The apparent temperature distribution map is corrected by temperature based on the emissivity to obtain a thermal distribution image.

[0068] Optionally, the abnormal circuit coordinate set is acquired based on the thermal distribution image, including:

[0069] The thermal distribution image is segmented by temperature threshold to obtain a plurality of abnormal pixels;

[0070] The plurality of abnormal pixels are clustered and analyzed to obtain a plurality of abnormal connected domains;

[0071] The plurality of abnormal connected domains are screened by using a preset area threshold to obtain a plurality of abnormal heating areas;

[0072] A plurality of circumscribed rectangles are acquired based on the plurality of abnormal heating areas, and a plurality of abnormal circuit coordinates are acquired by using the plurality of circumscribed rectangles;

[0073] The plurality of abnormal circuit coordinates are summarized to obtain an abnormal circuit coordinate set.

[0074] To achieve the above object, the application further provides a data cooperation system for self-detecting a camera based on multi-modal detection data, comprising:

[0075] An appearance image acquisition module is configured to classify and mark a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels;

[0076] Each of the plurality of marked panels is subjected to the following operations:

[0077] An appearance image of the marked panel is acquired to obtain an appearance image;

[0078] A panel appearance detection module is configured to acquire an appearance confidence score based on the appearance image;

[0079] aggregate the appearance confidence scores to obtain a plurality of appearance confidence scores;

[0080] identify a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores;

[0081] a panel display detection module configured to detect a display state of the plurality of appearance qualified panels to obtain a plurality of display confidence scores;

[0082] identify a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores;

[0083] an abnormal circuit identification module configured to perform the following operations on each of the plurality of display abnormal panels:

[0084] perform infrared thermal imaging collection on the display abnormal panel to obtain a thermal distribution image;

[0085] obtain an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates;

[0086] aggregate the abnormal circuit coordinate sets to obtain a plurality of abnormal circuit coordinate sets, wherein each abnormal circuit coordinate set corresponds to a display abnormal panel;

[0087] aggregate the plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels, and the plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels to obtain a detection report.

[0088] To solve the above problems, the present application also provides an electronic device, which comprises:

[0089] a memory configured to store at least one instruction; and

[0090] a processor configured to execute the instruction stored in the memory to implement the above-mentioned data coordination method for self-detecting a camera based on multi-modal detection data.

[0091] To solve the above problems, the present application also provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned data coordination method for self-detecting a camera based on multi-modal detection data.

[0092] To solve the problems in the background art, the present application classifies and labels a plurality of pre-confirmed OLED panels to obtain a plurality of labeled panels, and performs the following operations on each of the plurality of labeled panels: collecting appearance images of the labeled panel to obtain appearance images. The present application embodiment realizes accurate identification and differentiation of different types of panels by classifying and labeling a plurality of pre-confirmed OLED panels and collecting appearance images one by one, effectively improving the efficiency and accuracy of subsequent detection and processing. On this basis, the present application obtains appearance confidence scores based on the appearance images, aggregates the appearance confidence scores to obtain a plurality of appearance confidence scores, and confirms a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores. It can be seen that the present application embodiment realizes efficient and accurate screening of OLED appearance defects by quantifying confidence based on appearance images and automatically sorting qualified and abnormal panels, thereby improving the efficiency of subsequent quality detection. Next, the present application performs display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores, and confirms a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores. It can be seen that the present application embodiment realizes closed-loop screening of OLED panel appearance and display double defects by further quantifying display confidence and automatically distinguishing display qualified and abnormal products, thereby effectively improving the yield of qualified products. Further, the present application performs the following operations on each of the plurality of display abnormal panels: collecting infrared thermal images of the display abnormal panel to obtain a thermal distribution image, obtaining an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates, aggregating the abnormal circuit coordinate set to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to the display abnormal panel one by one. It can be seen that the present application embodiment realizes accurate tracing of defects by infrared thermal imaging and accurate positioning of abnormal circuit coordinates for display abnormal panels, thereby providing an efficient and reliable basis for subsequent maintenance. Next, the present application aggregates a plurality of abnormal circuit coordinate sets corresponding to a plurality of appearance abnormal panels, a plurality of display qualified panels, a plurality of display abnormal panels, and a plurality of display abnormal panels to obtain a detection report. It can be seen that the present application embodiment realizes comprehensive integration and traceability of OLED panel quality data by aggregating appearance, display, and abnormal circuit coordinate information into a detection report, thereby significantly improving the standardization of the quality inspection process and the pertinence of subsequent processing. Therefore, the present application can solve the problems of low efficient defect information extraction and server resource waste caused by uploading all data in OLED panel detection. BRIEF DESCRIPTION OF DRAWINGS

[0093] Figure 1 A flowchart of a data collaboration method based on multi-modal detection data self-detection camera according to an embodiment of the present application is shown in the figure;

[0094] Figure 2 A functional module diagram of a data collaboration system based on multi-modal detection data self-detection camera provided by an embodiment of the present application is shown in the figure.

[0095] Figure 3 A structural schematic diagram of an electronic device for implementing the data collaboration method based on multi-modal detection data self-detection camera provided by an embodiment of the present application is shown in the figure.

[0096] Legend:

[0097] 1, electronic device; 10, processor; 11, memory; 12, bus; 101, appearance image acquisition module; 102, panel appearance detection module; 103, panel display detection module; 104, abnormal circuit confirmation module.

[0098] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0099] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0100] The embodiment of the present application provides a data collaboration method based on multi-modal detection data self-detection camera. The execution subject of the data collaboration method based on multi-modal detection data self-detection camera includes but is not limited to at least one of the electronic devices which can be configured to execute the method provided by the embodiment of the present application, such as a server and a terminal. In other words, the data collaboration method based on multi-modal detection data self-detection camera can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster.

[0101] Referring to Figure 1 The figure shows a flowchart of the data collaboration method based on multi-modal detection data self-detection camera provided by an embodiment of the present application. In the embodiment, the data collaboration method based on multi-modal detection data self-detection camera includes:

[0102] S1, classifying and marking a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels.

[0103] It should be explained that the classifying and marking a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels includes:

[0104] Obtaining a plurality of panel images based on a plurality of OLED panels, wherein the OLED panel and the panel image are one-to-one corresponding;

[0105] performing the following operations on each of the plurality of panel images:

[0106] contour extraction on the panel image to obtain a contour image;

[0107] classification recognition on the contour image based on a pre-constructed panel model template library to obtain a panel model;

[0108] aggregating the panel models to obtain a plurality of panel models;

[0109] obtaining a plurality of marked panels based on the plurality of panel models.

[0110] Further, the OLED panel is an OLED screen to be detected. The panel image is a top view digital image obtained by shooting the OLED panel using an industrial camera. Contour extraction is a process of converting the panel image into a binary image containing only the contour lines of the OLED panel. The purpose is to identify and outline the external contour of the OLED panel and the edge of the internal opening in the panel image, so as to be compared with the standard contour template in the panel model template library. Optionally, the Canny operator is used as the contour extraction method. The contour image is a binary image obtained by contour extraction, in which the contour lines of the OLED panel and the internal opening are represented by white pixels, and the background is represented by black pixels. The standard contour template is a model of a digitized contour created in advance for each known model of qualified OLED panel. The panel model template library is a database storing standard contour templates of various known models of OLED panels. For example, the standard contour template of the Apple 14 series OLED panel, the standard contour template of the Apple 15 series OLED panel, and the standard contour template of the Apple 16 series OLED panel. Classification recognition is a process of comparing the contour image with all standard contour templates in the panel model template library. The purpose is to confirm the standard contour template with the highest similarity, and determine the model corresponding to the template as the model of the OLED panel. Optionally, a deep learning model is used as the classification recognition method. The panel model is a code identifier of the classification recognition, used to determine the product specification of the OLED panel. For example, 14 represents the Apple 14 series panel. The marked panel is a panel with a mark obtained by binding the panel model with the OLED panel corresponding thereto.

[0111] S2, performing the following operations on each of the plurality of marked panels: performing appearance image acquisition on the marked panel to obtain an appearance image.

[0112] It should be explained that the appearance image acquisition on the marked panel to obtain an appearance image includes:

[0113] obtaining a detection area definition parameter based on the marked panel;

[0114] image acquisition of the mark panel to obtain a mark panel image;

[0115] classification extraction of the mark panel image based on the detection area definition parameter to obtain a special-shaped structure image;

[0116] acquisition of a target panel image by using the mark panel image and the special-shaped structure image;

[0117] preprocessing of the target panel image to obtain an appearance image.

[0118] Further, the detection area definition parameter is a coordinate range parameter called from a pre-stored configuration database based on a panel model bound by the mark panel. The coordinate range parameter is used to confirm the positions and ranges of key areas in the mark panel, such as an effective display area (normal display screen area) and a special-shaped structure area (such as a camera hole and a notch area) that needs to be excluded. The configuration database is a database that stores key areas of mark panels of all known models. Image acquisition is a process of taking a photograph of the mark panel by using an industrial camera to obtain a digital image thereof. The mark panel image is a complete image obtained by image acquisition of the mark panel. Classification extraction is a process of separating different areas from the mark panel image according to the detection area definition parameter. Specifically, the image area representing the special-shaped structure area (such as a hole and a notch) is segmented by using the coordinate range defined in the detection area definition parameter. The purpose is to distinguish the non-defective special-shaped structure area from abnormal defects (such as scratches) that need to be detected in subsequent analysis, to prevent misjudgment. The special-shaped structure image is an image obtained after classification extraction, which only contains the special-shaped structure area (such as a camera hole).

[0119] Understandably, the target panel image is an image obtained by excluding the special-shaped structure image from the mark panel image. Preprocessing is a series of image enhancement operations performed on the target panel image, such as brightness equalization, geometric distortion correction, and image sharpening. The purpose is to optimize the image quality of the target panel image to facilitate more accurate subsequent defect analysis. The preprocessing is a prior art, which will not be described here. The appearance image is an image obtained after preprocessing of the target panel image.

[0120] S3, obtaining an appearance confidence score based on the appearance image, summarizing the appearance confidence score to obtain a plurality of appearance confidence scores, and confirming a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores.

[0121] It needs to be explained that the obtaining of the appearance confidence score based on the appearance image includes:

[0122] global scanning of the appearance image to obtain a gray scale distribution image;

[0123] confirming a defect pixel set from the gray scale distribution image and a pre-confirmed standard gray scale image, wherein one or more defect pixels exist in the defect pixel set;

[0124] constructing an initial defect position map based on the defect pixel set;

[0125] confirming a plurality of defect types and a plurality of defect areas from the initial defect position map and a pre-constructed defect screening model, wherein the defect types and the defect areas correspond to each other one by one;

[0126] calculating an appearance confidence score based on the plurality of defect types and the plurality of defect areas.

[0127] Further, the global scanning is a process of calculating the gray scale value of each pixel point in the appearance image. The gray scale value calculation is a prior art, which will not be described here. The gray scale scale image is an image obtained after the color appearance image is converted into a gray scale image, wherein the value of each pixel point represents its brightness (gray scale value), and the value range is 0 (black) to 255 (white). The standard gray scale image refers to a gray scale image corresponding to the current model of the OLED panel without any defects. It is used as a reference for judging whether the pixel point in the gray scale distribution image is abnormal. The defect pixel set is a set containing a plurality of defect pixels. The defect pixel refers to a pixel point in the gray scale distribution image, whose gray scale value deviates from the gray scale value of the pixel at the corresponding position in the standard gray scale image by greater than or equal to a gray scale value deviation threshold (for example, 10). The gray scale value deviation is the absolute value of the difference between the gray scale value of a pixel in the gray scale distribution image and the gray scale value of the pixel at the corresponding position in the standard gray scale image. The initial defect position map is a binary image with the same size as the appearance image, wherein all defect pixels in the defect pixel set are marked as 1 (white), and other pixels are marked as 0 (black). It is used to visually display the distribution position of all suspected defects.

[0128] It can be understood that the defect screening model is a deep learning model taking the initial defect position map as input and taking the defect type and the defect area in the initial defect position map as output, which is used to identify real defects (such as scratches and dents) and pseudo-defects such as imaging noise. Optionally, a convolutional neural network is used as the defect screening model. The defect type is a specific appearance defect category identified by the defect screening model, for example: scratches, dents, contamination, and uneven coating. The defect area is the number of pixel points occupied by the defect area of each defect type in the initial defect position map. For example, 5. The appearance confidence score is a quantitative index for representing the appearance quality of the OLED panel, wherein the higher the score, the better the appearance quality of the OLED panel.

[0129] It should be explained that the calculation of the appearance confidence score based on the plurality of defect types and the plurality of defect areas includes:

[0130] Multiple defect weights are obtained by using multiple defect types and a pre-built defect type-weight mapping table, where each defect weight corresponds one-to-one with a defect type;

[0131] Obtain the number of defect types and the area of ​​the detection region;

[0132] The appearance confidence score is calculated using multiple defect weights, multiple defect areas, the number of defect types, and the area of ​​the inspection region. The formula for calculating the appearance confidence score is as follows:

[0133] ,

[0134] in, This represents the appearance confidence score. Indicates the number of defect types. Represents the weight of the first defect among multiple defect weights. Each defect weight Represents the area of ​​multiple defects. Area of ​​each defect Indicates the area of ​​the detection region. This indicates taking the maximum value.

[0135] Furthermore, the defect type-weight mapping table is a predefined lookup table used to assign a weight coefficient to each defect type. For example, a dent has a defect weight of 0.9, and contamination has a defect weight of 0.3. The defect weight is a weight coefficient retrieved from the mapping table based on the defect type, used to characterize the severity of different defect types. A larger weight indicates a greater negative impact of that defect type on the appearance quality of the OLED panel. The number of defect types is the total number of defect types. For example, if scratches, dents, contamination, and uneven coating are present, the number of defect types is 4. The detection area is the number of pixels in the target panel image.

[0136] It should be explained that identifying multiple acceptable and unacceptable panels based on multiple appearance confidence scores means comparing each appearance confidence score with a preset appearance confidence score threshold. If the appearance confidence score is greater than or equal to the threshold, the panel corresponding to that score is considered acceptable, and these acceptable panels are aggregated to obtain multiple acceptable panels. Conversely, if the appearance confidence score is less than the threshold, the panel corresponding to that score is considered unacceptable, and these unacceptable panels are aggregated to obtain multiple unacceptable panels. Acceptable panels are those with appearance confidence scores greater than or equal to the threshold. Unacceptable panels are those with appearance confidence scores less than the threshold.

[0137] S4, performing display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores.

[0138] It should be explained that the display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores comprises:

[0139] The following operations are performed on each of the plurality of appearance qualified panels:

[0140] Four standard pure color pictures are obtained;

[0141] The following operations are performed on each of the four standard pure color pictures:

[0142] A lighted panel is obtained based on the appearance qualified panel and the standard pure color picture;

[0143] Image acquisition is performed on the lighted panel to obtain a display image;

[0144] Effective area extraction is performed on the display image to obtain an effective display area image;

[0145] Defect quantification processing is performed on the effective display area image to obtain a channel display score;

[0146] The channel display scores are summarized to obtain a plurality of channel display scores;

[0147] The display confidence scores are calculated based on the plurality of channel display scores;

[0148] The display confidence scores are summarized to obtain a plurality of display confidence scores.

[0149] Further, the standard pure color picture is a pre-set full-screen pure color image. For example, four basic colors of full white, full red, full green and full blue. The working state of each pixel unit of the appearance qualified panel is evaluated. The lit panel is the appearance qualified panel displayed according to the standard pure color picture. The display image is a color image obtained after image acquisition of the lit panel. The effective area extraction is a process of removing the non-display area (for example, the frame) from the display image according to the pre-set panel specification parameters. The purpose is to ensure that the analysis object is only the light-emitting pixel area. The effective display area image is an image obtained after effective area extraction. The image defect quantification process is a process of quantifying and scoring the effective display area image. The channel display score is a score given to the display quality of the appearance qualified panel under the standard pure color picture for the current test, wherein the higher the channel display score, the better the display quality under the standard pure color picture. The display confidence score is an overall display performance score obtained by averaging the channel display scores of the appearance qualified panel under the four standard pure color pictures. It is used to represent the degree of excellence of the display state of the appearance qualified panel.

[0150] It should be explained that the defect quantification process of the effective display area image obtains a channel display score, which includes:

[0151] The brightness of the effective display area image is counted to obtain the average brightness and the brightness standard deviation;

[0152] The average brightness and the pre-confirmed standard brightness are used to obtain the brightness offset degree;

[0153] The brightness offset degree and the brightness standard deviation are used to calculate the channel display score, wherein the calculation formula of the channel display score is as follows:

[0154] ,

[0155] wherein, the channel display score is represented by, the pre-set brightness deviation coefficient is represented by, the brightness offset degree is represented by, the pre-set maximum brightness offset degree is represented by, the pre-set uniformity index coefficient is represented by, the brightness standard deviation is represented by, the pre-set maximum brightness standard deviation is represented by, the minimum value is represented by.

[0156] Further, the brightness statistics is a process of calculating the average value of the brightness of all pixel points in the effective display area image and the standard deviation of the brightness, wherein the brightness statistics is a prior art, which will not be described here. The average brightness is the average value of the brightness of all pixel points in the effective display area image obtained by the brightness statistics. The brightness standard deviation is the standard deviation of the brightness of all pixel points in the effective display area image obtained by the brightness statistics. It is used to measure the uniformity of the brightness of all pixel points in the effective display area image. The standard brightness is the theoretical brightness target value that the qualified OLED panel should reach under the standard pure color picture. For example, 500 nits. The brightness deviation degree is the absolute value of the difference between the average brightness and the standard brightness, which is used to measure the deviation degree of the actual luminous brightness of the OLED panel from the theoretical target. The calculation formula of the brightness deviation degree is as follows:

[0157] ,

[0158] wherein, represents the average brightness, represents the standard brightness.

[0159] It can be understood that the brightness deviation coefficient is a coefficient for balancing the weight of the brightness accuracy in the channel display score. For example, 0.6. The maximum brightness deviation degree is the upper limit value of the acceptable brightness deviation degree. For example, 100. The uniformity index coefficient is a coefficient for balancing the weight of the brightness standard deviation in the channel display score. For example, 0.4. The maximum brightness standard deviation is the upper limit value of the acceptable brightness standard deviation. For example, 20.

[0160] S5, based on the plurality of display confidence scores, confirming a plurality of display qualified panels and a plurality of display abnormal panels.

[0161] It should be explained that based on the plurality of display confidence scores, the plurality of display qualified panels and the plurality of display abnormal panels are confirmed, which means that the plurality of display confidence scores are compared with the preset display confidence score threshold value, if the display confidence score is greater than or equal to the display confidence score threshold value, the appearance qualified panel corresponding to the display confidence score is regarded as the display qualified panel, the display qualified panels are summarized to obtain the plurality of display qualified panels; if the display confidence score is less than the display confidence score threshold value, the appearance qualified panel corresponding to the display confidence score is regarded as the display abnormal panel, and the display abnormal panels are summarized to obtain the plurality of display abnormal panels. The display qualified panel is the appearance qualified panel corresponding to the display confidence score greater than or equal to the display confidence score threshold value. The display abnormal panel is the appearance qualified panel corresponding to the display confidence score less than the display confidence score threshold value.

[0162] S6, performing the following operation on each of the plurality of display abnormal panels: performing infrared thermal imaging acquisition on the display abnormal panel to obtain a thermal distribution image.

[0163] It should be explained that the infrared thermal imaging acquisition on the display abnormal panel to obtain a thermal distribution image includes:

[0164] Based on the display abnormal panel, an infrared acquisition parameter is obtained.

[0165] Using the infrared acquisition parameter, infrared image acquisition is performed on the display abnormal panel according to a preset acquisition time interval to obtain a thermal image sequence.

[0166] The thermal image sequence is superimposed and filtered to reduce noise to obtain a denoising image.

[0167] Using the pre-confirmed blackbody reference data, the denoising image is calibrated for absolute temperature to obtain an apparent temperature distribution map.

[0168] The emissivity of the display abnormal panel is obtained.

[0169] Based on the emissivity, the apparent temperature distribution map is temperature corrected to obtain a thermal distribution image.

[0170] Further, the infrared acquisition parameter is a set of settings for controlling the infrared thermal imager based on the model of the display abnormal panel, which is called from a pre-stored configuration. For example, resolution, frame rate, temperature measurement range, focal length and lens field of view, to ensure that the thermal image collected can clearly cover the display abnormal panel. The acquisition time interval is the time interval between consecutive thermal image acquisitions. For example, 30 milliseconds. Infrared image acquisition is the process of using an infrared thermal imager and an acquisition time interval to take pictures of a display abnormal panel to obtain multiple thermal images. Optionally, a 16-bit infrared thermal imager is used as the device for the infrared image acquisition. The thermal image sequence is a set of thermal images arranged in time sequence obtained by infrared image acquisition. The thermal image is an image collected by an infrared thermal imager, wherein the gray value of each pixel in the thermal image corresponds to the infrared radiation intensity of the point on the surface of the display abnormal panel. Superimposed filtering and noise reduction is a signal processing technique that uses multiple thermal images in the thermal image sequence to suppress random noise. Specifically, the gray values of the same pixel in multiple thermal images at different acquisition times are arithmetically averaged to obtain a specific value, which is used as a new gray value. The multiple new gray values are summarized according to the positions of the pixels to obtain a denoising image. The purpose is to effectively suppress noise and improve the signal-to-noise ratio by superimposed averaging of multiple thermal images. The denoising image is a thermal image obtained by superimposed filtering and noise reduction.

[0171] It can be understood that the blackbody reference data is calibration data obtained by pre-aligning the infrared thermal imager to a temperature-controllable absolute blackbody and measuring at different temperature points. The blackbody reference data is used to confirm the correspondence between the gray value and the true temperature. Optionally, a standard blackbody radiation source with an emissivity greater than or equal to 0.95 is used as the absolute blackbody. The absolute temperature calibration is a process of converting the gray value of each pixel in the denoised image to the corresponding temperature by using the blackbody reference data. The apparent temperature distribution map is an image obtained after the absolute temperature calibration. Each pixel value in the apparent temperature distribution map represents a temperature calculated under the condition of the absolute blackbody.

[0172] For example, an absolute blackbody with accurate temperature control is placed in front of the infrared thermal imager, and the surface temperature of the absolute blackbody is stabilized at 30.0 The thermal image of the absolute blackbody is captured by using the infrared thermal imager. The average gray value of the thermal image is obtained, for example, 10000. The first calibration point is (30.0 , 10000). The surface temperature of the absolute blackbody is stabilized at 40.0 The thermal image of the absolute blackbody is captured by using the infrared thermal imager. The average gray value of the thermal image is obtained, for example, 15000. The second calibration point is (40.0 , 15000). The process is repeated to obtain multiple calibration points, and the multiple calibration points are summarized to obtain the blackbody reference data.

[0173] Further, the emissivity is an inherent thermal physical property of the surface of an object, and the emissivity of the OLED panel glass is 0.85. The temperature correction is a process of compensating and operating the apparent temperature distribution map according to the emissivity to obtain the true physical temperature. Optionally, the Stefan-Boltzmann law is used as the calculation method of the temperature correction. The thermal distribution image is an image representing the true temperature distribution of the display abnormal panel surface.

[0174] S7, obtaining an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates, and a plurality of abnormal circuit coordinate sets are obtained by summarizing the abnormal circuit coordinate set, wherein the abnormal circuit coordinate set corresponds to the display abnormal panel one by one.

[0175] It should be explained that the abnormal circuit coordinate set is obtained based on the thermal distribution image, including:

[0176] performing temperature threshold segmentation on the thermal distribution image to obtain a plurality of abnormal pixels;

[0177] performing clustering analysis on the plurality of abnormal pixels to obtain a plurality of abnormal connected domains;

[0178] The plurality of abnormal heat-emitting regions are obtained by screening the plurality of abnormal connected domains using a preset area threshold value;

[0179] A plurality of bounding rectangles are obtained based on the plurality of abnormal heat-emitting regions, and a plurality of abnormal circuit coordinates are obtained using the plurality of bounding rectangles;

[0180] The plurality of abnormal circuit coordinates are summarized to obtain an abnormal circuit coordinate set.

[0181] Further, the temperature threshold segmentation is a pixel value-based image segmentation method. Specifically, the temperature value of each pixel point in the heat distribution image is compared with a preset abnormal temperature threshold value, and if the temperature value is greater than or equal to the abnormal temperature threshold value, the pixel point corresponding to the temperature value is regarded as an abnormal pixel. The abnormal pixels are summarized to obtain a plurality of abnormal pixels. The abnormal pixel is a pixel point in the heat distribution image whose temperature value is greater than or equal to the abnormal temperature threshold value. The clustering analysis is a process of aggregating abnormal pixels with adjacent spatial positions together to obtain a plurality of pixel blocks. Optionally, a region growing method is used as the clustering analysis method. The abnormal connected domain is a region including a plurality of abnormal pixels obtained by clustering analysis. The area threshold value is a numerical value used to filter out abnormal connected domains with too small areas caused by noise. For example, 15 pixels. The abnormal heat-emitting region is an abnormal connected domain with an area greater than or equal to the area threshold value. The bounding rectangle is the smallest rectangle that exactly encloses the abnormal heat-emitting region. Optionally, a convex hull algorithm is used as the method for obtaining the bounding rectangle. The abnormal circuit coordinate is the pixel coordinate of the geometric center of the bounding rectangle in the heat distribution image, wherein the image pixel coordinate system is used as the coordinate system of the abnormal circuit coordinate, for example, (X, Y), wherein X represents the Xth column in the image pixel coordinate system, and Y represents the Yth row in the image pixel coordinate system. If the abnormal circuit coordinate is (3, 4), it means that the abnormal circuit coordinate is located in the third row and the fourth column of the heat distribution image. The abnormal circuit coordinate set is a set containing a plurality of abnormal circuit coordinates.

[0182] S8, a plurality of appearance abnormal panels, a plurality of display qualified panels, a plurality of display abnormal panels, and a plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels are summarized to obtain a detection report.

[0183] It should be explained that the detection report is a collection of data containing a plurality of appearance abnormal panels, a plurality of display qualified panels, a plurality of display abnormal panels, and a plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels. It is used to upload to a pre-confirmation server for subsequent processing.

[0184] To solve the problems in the background art, the present application classifies and labels a plurality of pre-confirmed OLED panels to obtain a plurality of labeled panels, and performs the following operations on each of the plurality of labeled panels: collecting appearance images of the labeled panel to obtain appearance images. The present application embodiment realizes accurate identification and differentiation of different types of panels by classifying and labeling a plurality of pre-confirmed OLED panels and collecting appearance images one by one, effectively improving the efficiency and accuracy of subsequent detection and processing. On this basis, the present application obtains appearance confidence scores based on the appearance images, aggregates the appearance confidence scores to obtain a plurality of appearance confidence scores, and confirms a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores. It can be seen that the present application embodiment realizes efficient and accurate screening of OLED appearance defects by quantifying confidence based on appearance images and automatically sorting qualified and abnormal panels, thereby improving the efficiency of subsequent quality detection. Next, the present application performs display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores, and confirms a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores. It can be seen that the present application embodiment realizes closed-loop screening of OLED panel appearance and display double defects by further quantifying display confidence and automatically distinguishing display qualified and abnormal products, thereby effectively improving the yield of qualified products. Further, the present application performs the following operations on each of the plurality of display abnormal panels: collecting infrared thermal images of the display abnormal panel to obtain a thermal distribution image, obtaining an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates, aggregating the abnormal circuit coordinate set to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to the display abnormal panel one by one. It can be seen that the present application embodiment realizes accurate tracing of defects by infrared thermal imaging and accurate positioning of abnormal circuit coordinates for display abnormal panels, thereby providing an efficient and reliable basis for subsequent maintenance. Next, the present application aggregates a plurality of abnormal circuit coordinate sets corresponding to a plurality of appearance abnormal panels, a plurality of display qualified panels, a plurality of display abnormal panels, and a plurality of display abnormal panels to obtain a detection report. It can be seen that the present application embodiment realizes comprehensive integration and traceability of OLED panel quality data by aggregating appearance, display, and abnormal circuit coordinate information into a detection report, thereby significantly improving the standardization of the quality inspection process and the pertinence of subsequent processing. Therefore, the present application can solve the problems of low efficient defect information extraction and server resource waste caused by uploading all data in OLED panel detection.

[0185] As Figure 2 shown, it is a functional module diagram of a data collaboration system based on multi-modal detection data self-detection camera provided by an embodiment of the present application.

[0186] The data cooperation system 100 based on multi-modal detection data from a camera can be installed in an electronic device. According to the functions implemented, the data cooperation system 100 based on multi-modal detection data from a camera can include an appearance image acquisition module 101, a panel appearance detection module 102, a panel display detection module 103, and an abnormal circuit confirmation module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0187] The appearance image acquisition module 101 is configured to classify and label a plurality of pre-confirmed OLED panels to obtain a plurality of labeled panels.

[0188] The following operations are performed on each of the plurality of labeled panels:

[0189] The appearance image acquisition module 101 is configured to classify and label a plurality of pre-confirmed OLED panels to obtain a plurality of labeled panels.

[0190] The panel appearance detection module 102 is configured to obtain an appearance confidence score based on the appearance image.

[0191] The appearance confidence scores are summarized to obtain a plurality of appearance confidence scores.

[0192] Based on the plurality of appearance confidence scores, a plurality of appearance qualified panels and a plurality of appearance abnormal panels are confirmed.

[0193] The panel display detection module 103 is configured to detect the display state of the plurality of appearance qualified panels to obtain a plurality of display confidence scores.

[0194] Based on the plurality of display confidence scores, a plurality of display qualified panels and a plurality of display abnormal panels are confirmed.

[0195] The abnormal circuit confirmation module 104 is configured to perform the following operations on each of the plurality of display abnormal panels:

[0196] The infrared thermal imaging acquisition module 104 is configured to acquire a thermal distribution image of the display abnormal panel.

[0197] The abnormal circuit confirmation module 104 is configured to obtain an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates.

[0198] The abnormal circuit confirmation module 104 is configured to summarize the abnormal circuit coordinate set to obtain a plurality of abnormal circuit coordinate sets, wherein each abnormal circuit coordinate set corresponds to a display abnormal panel.

[0199] The multiple appearance abnormal panels, multiple display qualified panels, multiple display abnormal panels and multiple abnormal circuit coordinate sets corresponding to the multiple display abnormal panels are aggregated to obtain a detection report.

[0200] In detail, the modules in the data collaboration system 100 for detecting the camera based on the multi-modal detection data in the embodiments of the present application adopt the same technical means as the data collaboration method for detecting the camera based on the multi-modal detection data in the above Figure 1 application, and can produce the same technical effects, which will not be described here in detail.

[0201] As shown in Figure 3 FIG. 1 is a structural schematic diagram of an electronic device for implementing the data collaboration method for detecting the camera based on the multi-modal detection data according to an embodiment of the present application.

[0202] The electronic device 1 can include a processor 10, a memory 11 and a bus 12, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a data collaboration method program for detecting the camera based on the multi-modal detection data.

[0203] The memory 11 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, for example, a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, for example, a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 includes an internal storage unit of the electronic device 1 and also includes an external storage device. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of the data collaboration method program for detecting the camera based on the multi-modal detection data, but also to temporarily store data that has been output or will be output.

[0204] The processor 10 can be composed of integrated circuits in some embodiments, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as a data collaboration method program for detecting the camera based on multi-modal detection data), and calls data stored in the memory 11, to perform various functions and process data of the electronic device 1.

[0205] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11, the at least one processor 10, etc.

[0206] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0207] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) for powering various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management system, so as to realize functions such as charge management, discharge management, and power consumption management through the power management system. The power supply can also include one or more direct current or alternating current power supplies, a recharging system, a power supply fault detection circuit, a power supply converter or inverter, a power supply status indicator, etc. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0208] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is usually used to establish a communication connection between the electronic device 1 and other electronic devices.

[0209] Optionally, the electronic device 1 can also include a user interface, which can be a display, an input unit such as a keyboard, and optionally, a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the electronic device 1 and for displaying a visualized user interface.

[0210] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0211] The data collaboration method program stored in the memory 11 in the electronic device 1 based on the multi-modal detection data self-detection camera is a combination of multiple instructions, which, when running in the processor 10, can realize:

[0212] Classifying and marking the pre-confirmed multiple OLED panels to obtain multiple marked panels;

[0213] For each marked panel in the multiple marked panels, the following operations are performed:

[0214] Collecting appearance images of the marked panel to obtain appearance images;

[0215] Obtaining appearance confidence scores based on the appearance images;

[0216] Summarizing the appearance confidence scores to obtain multiple appearance confidence scores;

[0217] Confirming multiple appearance qualified panels and multiple appearance abnormal panels based on the multiple appearance confidence scores;

[0218] Detecting display states of the multiple appearance qualified panels to obtain multiple display confidence scores;

[0219] Confirming multiple display qualified panels and multiple display abnormal panels based on the multiple display confidence scores;

[0220] For each display abnormal panel in the multiple display abnormal panels, the following operations are performed:

[0221] Collecting infrared thermal images of the display abnormal panel to obtain thermal distribution images;

[0222] Obtaining an abnormal circuit coordinate set based on the thermal distribution images, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates;

[0223] aggregate the abnormal circuit coordinate sets to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to one display abnormal panel;

[0224] aggregate the plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels, and the plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels to obtain a detection report.

[0225] Specifically, the specific implementation method of the processor 10 to the above instructions can refer to Figures 1 to 3 The description of related steps in the corresponding embodiments will not be repeated here.

[0226] Further, the modules / units integrated in the electronic device 1, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or system capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory).

[0227] The application also provides a computer readable storage medium, the readable storage medium stores a computer program, the computer program can realize the following when being executed by a processor of an electronic device:

[0228] classify and mark the pre-confirmed plurality of OLED panels to obtain a plurality of marked panels;

[0229] perform the following operations on each of the plurality of marked panels:

[0230] perform appearance image acquisition on the marked panel to obtain an appearance image;

[0231] obtain an appearance confidence score based on the appearance image;

[0232] aggregate the appearance confidence scores to obtain a plurality of appearance confidence scores;

[0233] confirm a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores;

[0234] perform display state detection on the plurality of appearance qualified panels to obtain a plurality of display confidence scores;

[0235] confirm a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores;

[0236] The following operations are performed on each of the plurality of display abnormal panels:

[0237] Infrared thermal imaging acquisition is performed on the display abnormal panel to obtain a thermal distribution image.

[0238] The abnormal circuit coordinate set is obtained based on the thermal distribution image, wherein the abnormal circuit coordinate set includes one or more abnormal circuit coordinates.

[0239] The abnormal circuit coordinate sets are summarized to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate sets correspond to the display abnormal panels one by one.

[0240] The plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels, and the plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels are summarized to obtain a detection report.

[0241] In several embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the above-described system embodiments are only illustrative, and actual implementation can have another division manner.

[0242] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0243] In addition, the functional modules in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software functional modules.

[0244] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0245] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or systems stated in the system claims can also be implemented by one unit or system through software or hardware. The second word is used to indicate the name, and does not indicate any specific order.

[0246] Finally, it should be noted that the above examples are merely intended to illustrate the technical solutions of the present application and not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A data collaboration method for self-detecting a camera based on multi-modal detection data, characterized in that, The method comprises: Classifying and marking a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels; For each of the plurality of marked panels, the following operations are performed: Collecting appearance images of the marked panel to obtain appearance images; Obtaining appearance confidence scores based on the appearance images; Summarizing the appearance confidence scores to obtain a plurality of appearance confidence scores; Confirming a plurality of appearance qualified panels and a plurality of appearance abnormal panels based on the plurality of appearance confidence scores; Detecting display states of the plurality of appearance qualified panels to obtain a plurality of display confidence scores; Confirming a plurality of display qualified panels and a plurality of display abnormal panels based on the plurality of display confidence scores, wherein confirming the plurality of display qualified panels and the plurality of display abnormal panels based on the plurality of display confidence scores means comparing each of the plurality of display confidence scores with a preset display confidence score threshold value, if the display confidence score is greater than or equal to the display confidence score threshold value, the appearance qualified panel corresponding to the display confidence score is regarded as a display qualified panel, and the display qualified panels are summarized to obtain the plurality of display qualified panels, if the display confidence score is less than the display confidence score threshold value, the appearance qualified panel corresponding to the display confidence score is regarded as a display abnormal panel, and the display abnormal panels are summarized to obtain the plurality of display abnormal panels; For each of the plurality of display abnormal panels, the following operations are performed: Collecting infrared thermal images of the display abnormal panel to obtain a thermal distribution image; The collecting of the infrared thermal images of the display abnormal panel to obtain the thermal distribution image comprises: Obtaining infrared collection parameters based on the display abnormal panel; Collecting infrared images of the display abnormal panel according to a preset collection time interval by using the infrared collection parameters to obtain a thermal image sequence; Performing superposition filtering and noise reduction on the thermal image sequence to obtain a denoised image; Calibrating the denoised image by using pre-confirmed blackbody reference data to obtain an apparent temperature distribution map; Obtaining an emission value rate of the display abnormal panel; Performing temperature correction on the apparent temperature distribution map based on the emission value rate to obtain a thermal distribution image, wherein the thermal distribution image is an image representing the thermal distribution of the circuit under the abnormal panel; Obtaining an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set comprises one or more abnormal circuit coordinates; The obtaining of the abnormal circuit coordinate set based on the thermal distribution image comprises: Performing temperature threshold segmentation on the thermal distribution image to obtain a plurality of abnormal pixels; Performing clustering analysis on the plurality of abnormal pixels to obtain a plurality of abnormal connected domains; Filtering the plurality of abnormal connected domains by using a preset area threshold to obtain a plurality of abnormal heating areas; Obtaining a plurality of circumscribed rectangles based on the plurality of abnormal heating areas, and obtaining a plurality of abnormal circuit coordinates by using the plurality of circumscribed rectangles; Summarizing the plurality of abnormal circuit coordinates to obtain an abnormal circuit coordinate set; Summarizing the abnormal circuit coordinate set to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to the display abnormal panel in one-to-one correspondence; The multiple appearance abnormal panels, the multiple display qualified panels, the multiple display abnormal panels, and multiple abnormal circuit coordinate sets corresponding to the multiple display abnormal panels are aggregated to obtain a detection report. 2.The data collaboration method of self-detecting camera based on multi-modal detection data according to claim 1, wherein, The multiple OLED panels are classified and marked based on the pre-confirmation to obtain multiple marked panels, including: Multiple panel images are obtained based on the multiple OLED panels, wherein the OLED panels and the panel images correspond to each other in a one-to-one manner. The following operations are performed on each of the multiple panel images: Contour extraction is performed on the panel image to obtain a contour image. The contour image is classified and recognized based on a pre-constructed panel model template library to obtain a panel model. Multiple panel models are aggregated to obtain multiple panel models. Multiple marked panels are obtained based on the multiple panel models. 3.The data collaboration method of self-detecting camera based on multi-modal detection data according to claim 2, wherein, The following operations are performed on the marked panel to obtain an appearance image, including: Detection region definition parameters are obtained based on the marked panel. An image of the marked panel is captured to obtain a marked panel image. An abnormal shape structure image is obtained by classifying and extracting the marked panel image based on the detection region definition parameters. A target panel image is obtained by using the marked panel image and the abnormal shape structure image. The target panel image is preprocessed to obtain the appearance image. 4.The method of claim 3, wherein, The following operations are performed based on the appearance image to obtain an appearance confidence score, including: A global scan is performed on the appearance image to obtain a gray scale distribution image. A defective pixel set is confirmed by using the gray scale distribution image and a pre-confirmed standard gray scale image, wherein one or more defective pixels are present in the defective pixel set. An initial defect position map is constructed based on the defective pixel set. Multiple defect types and multiple defect areas are confirmed by using the initial defect position map and a pre-constructed defect screening model, wherein the defect types and the defect areas correspond to each other in a one-to-one manner. The appearance confidence score is calculated based on the multiple defect types and the multiple defect areas. 5.The data collaboration method of self-detecting camera based on multi-modal detection data according to claim 4, wherein, The following operations are performed to calculate the appearance confidence score based on the multiple defect types and the multiple defect areas, including: Multiple defect weights are obtained by using the multiple defect types and a pre-constructed defect type-weight mapping table, wherein the defect weights correspond to the defect types in a one-to-one manner. A defect type quantity and a detection region area are obtained. The appearance confidence score is calculated by using the multiple defect weights, the multiple defect areas, the defect type quantity, and the detection region area, wherein the calculation formula of the appearance confidence score is as follows: ， wherein, represents the appearance confidence score, represents the number of defect types, represents the i-th defect weight of the plurality of defect weights, represents the i-th defect area of the plurality of defect areas, represents the detection area, represents taking the maximum value.​​ 6.The data collaboration method of self-detecting camera based on multi-modal detection data according to claim 5, wherein, The following operations are performed to obtain multiple display confidence scores by detecting the display state of the multiple appearance qualified panels, including: The following operations are performed on each of the multiple appearance qualified panels: Four standard pure color pictures are obtained. The following operations are performed on each of the four standard pure color pictures: A lighted panel is obtained based on the appearance qualified panel and the standard pure color picture. An image of the lighted panel is captured to obtain a display image. An effective display region image is obtained by performing effective region extraction on the display image. A channel display score is obtained by performing defect quantization processing on the effective display region image. Multiple channel display scores are aggregated to obtain multiple channel display scores. A display confidence score is calculated based on the multiple channel display scores. Multiple display confidence scores are aggregated to obtain multiple display confidence scores. 7.The data collaboration method of self-detecting camera based on multi-modal detection data according to claim 6, wherein, The defect quantification processing on the effective display area image obtains a channel display score, and the defect quantification processing comprises the following steps: Statistically obtaining the average brightness and the brightness standard deviation of the effective display area image; Obtaining the brightness offset degree by using the average brightness and the pre-confirmed standard brightness; Calculating the channel display score by using the brightness offset degree and the brightness standard deviation, wherein the calculation formula of the channel display score is as follows: ， wherein, represents a channel display score, represents a preset luminance deviation coefficient, represents a luminance shift degree, represents a preset maximum luminance shift degree, represents a preset uniformity index coefficient, represents a luminance standard deviation, represents a preset maximum luminance standard deviation, represents a minimum value.

8. A system for using the data collaboration method from the camera based on multi-modal detection data according to any one of claims 1 to 7, characterized in that, The system comprises: An appearance image acquisition module, configured to classify and mark a plurality of pre-confirmed OLED panels to obtain a plurality of marked panels; Each of the plurality of marked panels is subjected to the following operations: Acquiring the appearance image of the marked panel to obtain an appearance image; A panel appearance detection module, configured to obtain an appearance confidence score based on the appearance image; The appearance confidence scores are summarized to obtain a plurality of appearance confidence scores; Based on the plurality of appearance confidence scores, a plurality of appearance qualified panels and a plurality of appearance abnormal panels are confirmed; A panel display detection module, configured to detect the display state of the plurality of appearance qualified panels to obtain a plurality of display confidence scores; Based on the plurality of display confidence scores, a plurality of display qualified panels and a plurality of display abnormal panels are confirmed; An abnormal circuit confirmation module, configured to perform the following operations on each of the plurality of display abnormal panels: Acquiring the thermal distribution image of the display abnormal panel by infrared thermal imaging acquisition to obtain a thermal distribution image; Obtaining an abnormal circuit coordinate set based on the thermal distribution image, wherein the abnormal circuit coordinate set comprises one or more abnormal circuit coordinates; Summarizing the abnormal circuit coordinate sets to obtain a plurality of abnormal circuit coordinate sets, wherein the abnormal circuit coordinate set corresponds to the display abnormal panel in one-to-one correspondence; Summarizing the plurality of appearance abnormal panels, the plurality of display qualified panels, the plurality of display abnormal panels and the plurality of abnormal circuit coordinate sets corresponding to the plurality of display abnormal panels to obtain a detection report.

Images