Method, device, equipment and medium for detecting panel support column

Abstract

The present disclosure relates to a panel support column detection method, device, equipment and medium. The method comprises: acquiring a first image of a panel support column to be detected; generating a second image of the panel support column to be detected based on the first image through a pre-trained super-resolution model, wherein the first resolution of the first image is less than the second resolution of the second image, the super-resolution model is used to generate an image with the second resolution, and the training data of the super-resolution model comprises a target image acquired at the second resolution via an image acquisition device; and determining a detection result of the panel support column to be detected based on the second image. Thus, the detection accuracy of the panel support column can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of industrial machine vision and intelligent inspection, and more specifically, to a method, apparatus, equipment, and medium for detecting panel support columns. Background Technology

[0002] Panel products often contain regularly arranged arrays of PS (Panel Support) pillars for support, gap control, or structural positioning. In AOI (Automated Optical Inspection Image) inspection, the appearance, edge morphology, and local texture of the PS pillars are crucial for anomaly identification. However, due to limitations in production line imaging speed, field of view, and hardware costs, actual inspection often uses lower resolution images (e.g., 5, 4, or 3 micrometers / pixel), resulting in blurred PS pillar boundaries, insufficient detail, and unstable positioning, thus affecting the reliability of subsequent anomaly detection.

[0003] In related technologies, some generative models are prone to introducing high-frequency details (i.e., illusions) that are inconsistent with the input observations. In industrial inspection scenarios, this can directly lead to false detections or missed detections, resulting in low detection accuracy of panel support columns. Summary of the Invention

[0004] The main purpose of this disclosure is to provide a method, apparatus, equipment and medium for detecting panel support columns, so as to solve the technical problem of low detection accuracy of panel support columns in the prior art, and to achieve the technical effect of improving the detection accuracy of panel support columns.

[0005] To achieve the above objectives, a first aspect of this disclosure provides a method for detecting panel support columns, comprising: Acquire the first image of the support column of the panel to be inspected; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. Based on the second image, the detection results of the support column of the panel to be detected are determined.

[0006] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Determine the magnification scale information of the second resolution of the second image relative to the first resolution of the first image; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image and magnification scale information.

[0007] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, along with uncertainty information of the second image; Based on the second image, the detection results of the support column of the panel to be detected are determined, including: Based on the second image and uncertainty information, the detection result of the support column of the panel to be detected is determined.

[0008] In some possible implementations, the detection result of the support column of the panel to be detected is determined based on the second image and uncertainty information, including: Based on uncertainty information, the abnormal threshold of the support column of the panel to be detected is determined; Based on the second image and the anomaly threshold, the detection results of the support column of the panel to be detected are determined.

[0009] In some possible implementations, the super-resolution model is trained as follows: Acquire a fourth image, and a target image acquired by an image acquisition device at a second resolution, wherein the resolution of the fourth image is lower than the second resolution; A super-resolution model is trained based on the fourth image and the target image.

[0010] In some possible implementations, a super-resolution model is trained based on the fourth image and the target image, including: The fourth image is input into the initial super-resolution model to obtain a reconstructed image at the second resolution. Based on the reconstructed image and the target image, determine the first loss value; A third image corresponding to the reconstructed image is generated using a pre-trained degradation model, wherein the degradation model is used to generate a third image with a resolution lower than that of the reconstructed image. Based on the fourth and third images, determine the second loss value; Upsample the fourth image to obtain an upsampled image with a second resolution; The third loss value is determined based on the edge information of the upsampled image and the edge information of the reconstructed image; Based on the first loss value, the second loss value, and the third loss value, the model parameters of the initial super-resolution model are adjusted to obtain the trained super-resolution model.

[0011] In some possible implementations, a first loss value is determined based on the reconstructed image and the target image, including: Based on the reconstructed image and the target image, determine the error information of the reconstructed image relative to the target image; Based on the error information and the uncertainty information of the reconstructed image, the first loss value is determined.

[0012] Secondly, embodiments of this disclosure provide a detection device for panel support columns, the device comprising: The acquisition unit is configured to: acquire a first image of the support column of the panel to be detected; The generation unit is configured to: generate a second image of the panel support column to be detected based on a first image using a pre-trained super-resolution model, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. The determining unit is configured to: determine the detection result of the support column of the panel to be detected based on the second image.

[0013] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Determine the magnification scale information of the second resolution of the second image relative to the first resolution of the first image; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image and magnification scale information.

[0014] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, along with uncertainty information of the second image; Based on the second image, the detection results of the support column of the panel to be detected are determined, including: Based on the second image and uncertainty information, the detection result of the support column of the panel to be detected is determined.

[0015] In some possible implementations, the detection result of the support column of the panel to be detected is determined based on the second image and uncertainty information, including: Based on uncertainty information, the abnormal threshold of the support column of the panel to be detected is determined; Based on the second image and the anomaly threshold, the detection results of the support column of the panel to be detected are determined.

[0016] In some possible implementations, the super-resolution model is trained as follows: Acquire a fourth image, and a target image acquired by an image acquisition device at a second resolution, wherein the resolution of the fourth image is lower than the second resolution; A super-resolution model is trained based on the fourth image and the target image.

[0017] In some possible implementations, a super-resolution model is trained based on the fourth image and the target image, including: The fourth image is input into the initial super-resolution model to obtain a reconstructed image at the second resolution. Based on the reconstructed image and the target image, determine the first loss value; A third image corresponding to the reconstructed image is generated using a pre-trained degradation model, wherein the degradation model is used to generate a third image with a resolution lower than that of the reconstructed image. Based on the fourth and third images, determine the second loss value; Upsample the fourth image to obtain an upsampled image with a second resolution; The third loss value is determined based on the edge information of the upsampled image and the edge information of the reconstructed image; Based on the first loss value, the second loss value, and the third loss value, the model parameters of the initial super-resolution model are adjusted to obtain the trained super-resolution model.

[0018] In some possible implementations, a first loss value is determined based on the reconstructed image and the target image, including: Based on the reconstructed image and the target image, determine the error information of the reconstructed image relative to the target image; Based on the error information and the uncertainty information of the reconstructed image, the first loss value is determined.

[0019] Thirdly, embodiments of this disclosure provide an electronic device, including: Memory, used to store computer programs; A processor is configured to execute a computer program stored in the memory, and when the computer program is executed, to implement the method of any embodiment of the panel support column detection method of the first aspect of this disclosure.

[0020] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any embodiment of the panel support column detection method of the first aspect described above.

[0021] Fifthly, embodiments of this disclosure provide a computer program including computer-readable code, wherein when the computer program instructions are executed by a processor, they implement the method of any embodiment of the panel support column detection method of the first aspect described above.

[0022] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In this disclosure, a super-resolution model is trained using a target image acquired at a second resolution by an image acquisition device. The super-resolution model is then used to perform super-resolution reconstruction on a low-resolution first image to obtain a high-resolution second image. This reduces the illusions present in the second image. Based on the second image, panel support columns are detected, which reduces the probability of false detection and false negative detection and improves the detection accuracy of panel support columns. Attached Figure Description

[0023] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of the disclosure and to make other features, objects, and advantages of the disclosure more apparent. The illustrative embodiments of the disclosure, along with their descriptions, are used to explain the disclosure and do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart illustrating a method for detecting panel support columns provided in this embodiment of the disclosure; Figure 2 A flowchart of another method for detecting panel support columns provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating another method for detecting panel support columns provided in this embodiment of the present disclosure; Figure 4 A flowchart illustrating another method for detecting panel support columns provided in this embodiment of the disclosure; Figure 5 A schematic flowchart illustrating the degradation model and degradation consistency constraints in a panel support column detection method provided in this embodiment of the present disclosure; Figure 6 A flowchart illustrating the addition of an edge penalty term in a panel support column detection method provided in this embodiment of the disclosure; Figure 7 This is a schematic diagram of the structure of the super-resolution model and the multi-task head in a panel support column detection method provided in an embodiment of this disclosure; Figure 8 A schematic diagram of the back projection verification and risk classification output process in the detection method of panel support column provided in this embodiment of the present disclosure; Figure 9 A schematic diagram of a detection device for a panel support column provided in an embodiment of this disclosure; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] In this disclosure, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are primarily for the purpose of better describing this disclosure and its embodiments, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to be constructed and operated in a specific orientation.

[0027] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in certain circumstances to indicate a dependency or connection. Those skilled in the art can understand the specific meaning of these terms in this disclosure according to the specific circumstances.

[0028] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0029] Figure 1 This is a flowchart illustrating a method for detecting a panel support column according to an embodiment of this disclosure. This method can be applied to one or more electronic devices such as smartphones, laptops, desktop computers, portable computers, and servers. Furthermore, the execution entity of this method can be hardware or software. When the execution entity is hardware, it can be one or more of the aforementioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the execution entity is software, this method can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here.

[0030] like Figure 1 As shown, the method specifically includes: Step 101: Obtain the first image of the support column of the panel to be detected.

[0031] In this embodiment, the panel support column to be tested can be the panel support column to be tested. As an example, the panel support column to be tested can be checked for abnormal height, cracks, or scratches, etc.

[0032] Panel support pillars can be structures used for support in panel products. For example, panel support pillars can be interlayer structures used to support the panel in a display panel, or support pillars used to position the chip relative to the substrate in a semiconductor device.

[0033] The first image can be any image acquired by the image acquisition device from the support column of the panel to be inspected. The resolution of the first image can be referred to as the first resolution. For example, the first resolution could be 3 micrometers / pixel, 4 micrometers / pixel, 5 micrometers / pixel, etc.

[0034] Step 102: Based on the first image, generate a second image of the support column of the panel to be detected using a pre-trained super-resolution model, wherein the first resolution of the first image is less than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device.

[0035] In this embodiment, the pre-trained super-resolution model can be a model that, after training, can convert a low-resolution (i.e., first resolution) first image into a high-resolution (i.e., second resolution) second image. Here, low resolution and high resolution are relative. In other words, the resolution of the first image is lower than the resolution of the second image.

[0036] In some alternative implementations, the super-resolution model may include structures such as Transformer, Convolutional Neural Network (CNN).

[0037] The second image can be an image of the panel support pillar to be detected, generated by a pre-trained super-resolution model based on the first image and having a second resolution. As an example, the first image can be an image of the panel support pillar to be detected with a resolution of 3 micrometers / pixel. The second image can be an image of the panel support pillar to be detected with a resolution of 2 micrometers / pixel.

[0038] The second resolution can be the resolution of the second image. As an example, the second resolution could be 2 micrometers per pixel.

[0039] Furthermore, the second image can be an image generated by the super-resolution model during the inference phase, and the reconstructed image can be an image generated by the super-resolution model during the training phase.

[0040] The target image can be an image acquired at a second resolution via an image acquisition device.

[0041] In some alternative implementations, a super-resolution model can be obtained by training multiple training data, including the target image (e.g., an image with a resolution of 2 micrometers / pixel) and the corresponding low-resolution images (e.g., 5, 4, or 3 micrometers / pixel), through the backpropagation algorithm.

[0042] In other alternative implementations, a super-resolution model can be obtained by training multiple sets of data, including the target image (e.g., an image with a resolution of 2 micrometers / pixel), corresponding low-resolution images (e.g., 3, 4, or 5 micrometers / pixel), and magnification scale information (e.g., 1.5x, 2x, or 2.5x), using a backpropagation algorithm. The magnification scale information can represent the resolution magnification factor of the second image relative to the first image.

[0043] In some optional implementations of this embodiment, the super-resolution model can also be trained in the following manner: The first step involves acquiring the fourth image (i.e., the low-resolution image corresponding to the target image) and the target image acquired by the image acquisition device at a second resolution. The resolution of the fourth image is lower than that of the second image.

[0044] The fourth image can be acquired via an image acquisition device or by processing a higher resolution image (e.g., a target image) (e.g., a degradation process to convert a high resolution image into a low resolution image).

[0045] An image acquisition device can be any device with image acquisition capabilities. As an example, an image acquisition device can be an industrial camera.

[0046] The target image can be an image acquired by an image acquisition device at a second resolution, and the target image can be used as the ground truth image in the training process of the super-resolution model.

[0047] The second step involves training a super-resolution model based on the fourth image and the target image.

[0048] In some alternative implementations, a fourth image can be input into an initial super-resolution model (which may include a Transformer structure, a convolutional neural network, or other structures) to obtain a reconstructed image at a second resolution. Then, based on the reconstructed image and the target image, a preset loss function is calculated. Using the backpropagation algorithm, the model parameters of the initial super-resolution model are adjusted based on the aforementioned function value, thereby obtaining a trained super-resolution model.

[0049] Here, the resolution of the reconstructed image is equal to the resolution of the second image. For example, the resolution of the reconstructed image (i.e., the second resolution) could be 2 micrometers per pixel.

[0050] It is understandable that training a super-resolution model based on the fourth image and the target image enables the super-resolution model to learn the mapping relationship from low-resolution images to high-resolution images, ensuring that the super-resolution model has the ability to convert the first image into a high-resolution second image, providing a more reliable second image for the subsequent detection process, thereby improving the detection accuracy of the panel support column.

[0051] In some application scenarios among the above optional implementation methods, the second step can also be implemented in the following ways: The first step is to input the fourth image into the initial super-resolution model to obtain a reconstructed image at the second resolution.

[0052] The initial super-resolution model can be an untrained super-resolution model or a trained model that has not met the preset training termination conditions. The initial super-resolution model can include structures such as Transformer and convolutional neural networks.

[0053] The reconstructed image can be an image with a second resolution output by the initial super-resolution model after the fourth image is input into it.

[0054] The second step is to determine the first loss value based on the reconstructed image and the target image.

[0055] The first loss value can represent the difference between the reconstructed image and the target image at the second resolution.

[0056] In some alternative implementations, the first loss value can be obtained by calculating the difference between the pixel values ​​of the reconstructed image and the target image; or, the first loss value can be calculated based on the structural similarity index measure (SSIM) between the reconstructed image and the target image.

[0057] In some alternative implementations, the first loss value can be determined based on the reconstructed image and the target image using at least one of the following loss functions: L1 (L1 Loss Function, also known as Mean Absolute Error (MAE)), Charbonnier (a smoothed L1 loss function), and SSIM (Structural Similarity Index Measure).

[0058] The third step involves generating a third image corresponding to the reconstructed image using a pre-trained degradation model.

[0059] The degradation model is used to generate an image with a lower resolution than the reconstructed image (i.e., a third image). In other words, the degradation model can be used to convert a high-resolution reconstructed image into a low-resolution third image. In some cases, the degradation model can also be used to simulate the degradation process in actual imaging.

[0060] In some alternative implementations, the degradation model may include point spread function convolution, downsampling operators, noise injection operators, etc.

[0061] In some alternative implementations, a backpropagation algorithm can be used to train a degradation model based on a high-resolution image and its corresponding low-resolution image. The high-resolution image and the corresponding low-resolution image can be images of different resolutions acquired using an image acquisition device from the same area of ​​the same panel support column.

[0062] Therefore, by generating a third image with a resolution lower than the second resolution through a pre-trained degradation model, and ensuring that the resolution of the third image is consistent with the first resolution of the first image, the comparison between the third image and the first image has the same resolution basis, making the comparison results more valuable. Furthermore, it can avoid introducing additional false features during the degradation process, ensuring the authenticity of the third image, and thus making the detection results generated based on the third image and the first image more accurate.

[0063] The third image can be generated by the degradation model after degrading the reconstructed image. The resolution of the third image is lower than that of the reconstructed image.

[0064] The fourth step is to determine the second loss value based on the fourth and third images.

[0065] The second loss value can be used to measure the difference between the degraded reconstructed image and the input low-resolution image (the fourth image); or, the second loss value can be used to measure the difference between the panel support pillar region in the degraded reconstructed image and the panel support pillar region in the input low-resolution image (the fourth image). The second loss value can be used to constrain the consistency between the third image obtained after degrading the reconstructed image using the degradation model and the corresponding input low-resolution image (i.e., the fourth image).

[0066] In some alternative implementations, the second loss value can be determined based on the fourth and third images using at least one of the following loss functions: L1, Charbonnier, SSIM, etc.

[0067] The fifth step is to upsample the fourth image to obtain an upsampled image with a second resolution.

[0068] The upsampled image can be an image with a second resolution obtained by upsampling the fourth image.

[0069] In some alternative implementations, a bilinear interpolation algorithm can be used to upsample the fourth image to obtain an upsampled image with a second resolution; alternatively, a nearest neighbor interpolation algorithm can be used to upsample the fourth image to obtain an upsampled image with a second resolution.

[0070] The sixth step is to determine the third loss value based on the edge information of the upsampled image and the edge information of the reconstructed image.

[0071] Among them, the edge information of the upsampled image can be the edge contour, edge intensity and other feature information of the support column of the panel to be detected in the upsampled image.

[0072] The edge information of the reconstructed image can be the edge contour, edge intensity, and other feature information of the support column of the panel to be detected in the reconstructed image.

[0073] The third loss value can be a loss value determined based on the difference between the edge information of the upsampled image and the edge information of the reconstructed image. The third loss value can be used to suppress high-frequency edge structures in the reconstructed image that are not supported by the fourth image.

[0074] In some optional implementations, the edge information of the upsampled image can be its edge intensity map, and the edge information of the reconstructed image can be its edge intensity map. Then, based on the edge intensity maps of the upsampled and reconstructed images, an edge energy increment map is obtained by subtracting each pixel. The difference between each pixel value in the edge energy increment map and a preset threshold is determined as a third loss value. This penalizes the portion of edge energy added to the reconstructed image that is not supported by the input.

[0075] The seventh step involves adjusting the model parameters of the initial super-resolution model based on the first, second, and third loss values ​​to obtain the trained super-resolution model.

[0076] The model parameters of the initial super-resolution model can be various parameters that constitute the initial super-resolution model (such as weights, biases, etc.).

[0077] In some alternative implementations, the model parameters can be adjusted using the backpropagation algorithm based on the sum or weighted sum of the first, second, and third loss values. The model parameters of the initial super-resolution model can be adjusted through multiple iterations. When the sum or weighted sum converges to a preset threshold, the trained super-resolution model is obtained.

[0078] It is understandable that the first loss value can represent the difference between the reconstructed image and the ground truth image at the second resolution, the second loss value can be used to constrain the consistency between the third image obtained after the reconstructed image is degraded by the degradation model and the corresponding input low-resolution image (fourth image), and the third loss value can be used to suppress high-frequency edge structures in the reconstructed image that are not supported by the input low-resolution image (fourth image). Thus, in the above application scenario, the performance of the super-resolution model is optimized from three dimensions: "the reconstructed image is close to the ground truth", "the degradation is consistent with the input", and "the edge structure is real". This enables the trained super-resolution model to effectively suppress false details while improving image resolution, ensuring the reliability of the reconstruction results, and thus improving the detection accuracy of panel support columns.

[0079] In some of the above application scenarios, the first loss value can be determined based on the reconstructed image and the target image using the following method: The first step is to determine the error information of the reconstructed image relative to the target image based on the reconstructed image and the target image.

[0080] Error information can represent the difference between the reconstructed image and the target image. For example, error information can be pixel-level difference information between the reconstructed image and the target image, containing the difference between the reconstructed value and the ground truth value for each pixel.

[0081] In some alternative implementations, error information can be obtained by calculating the pixel difference between the reconstructed image and the target image, or by calculating the structural similarity between the two and the peak signal-to-noise ratio (PSNR).

[0082] The second step is to determine the first loss value based on the error information and the uncertainty information of the reconstructed image.

[0083] The uncertainty information of the reconstructed image can represent the reliability of the reconstructed image generated by the super-resolution model; or, the reliability of each pixel of the reconstructed image.

[0084] In some alternative implementations, the super-resolution model can learn uncertainty information through heteroscedasticity regression; or, uncertainty information can be obtained based on the pixel error statistics between the reconstructed image and the target image.

[0085] It is understandable that by determining the error information of the reconstructed image relative to the target image, and then determining the first loss value based on the error information and uncertainty information, the calculation of the first loss value can be combined with the reliability of the reconstruction result, thereby improving the accuracy of the super-resolution model in generating the second image.

[0086] In some alternative implementations, the first image can be input into a pre-trained super-resolution model, which then generates a second image of the support column of the panel to be detected through operations such as feature extraction and super-resolution reconstruction.

[0087] In some alternative implementations, feature data can be obtained by first extracting features from the first image, and then the feature data can be input into a pre-trained super-resolution model, which can generate a second image of the support column of the panel to be detected through super-resolution reconstruction and other operations.

[0088] Step 103: Based on the second image, determine the detection result of the support column of the panel to be detected.

[0089] In this embodiment, the detection result can be the result obtained after detecting the state of the support column of the panel to be detected. As an example, the detection result may include at least one of the following: the position of the support column, whether there is an abnormal height, the center position coordinates of the support column, and the probability of an abnormality of the support column.

[0090] In some optional implementations, the detection result can be determined by performing feature analysis on the support column of the panel to be detected in the second image using a preset detection algorithm.

[0091] In some alternative implementations, the detection result can also be determined by processing the second image using a pre-trained detection task head.

[0092] In some alternative implementations, the height, edge shape, local texture, etc. of the support column of the panel to be detected can be determined based on the second image to determine whether they meet the preset standards, thereby determining the detection result of the support column of the panel to be detected.

[0093] In some alternative implementations, the second image can be analyzed using a high anomaly regression / classification task head to determine the detection result of the panel support column to be detected; alternatively, the second image can be analyzed using a defect detection algorithm to determine the detection result of the panel support column to be detected.

[0094] Furthermore, if the detection results indicate that there is an abnormality in the support column of the panel to be detected, a third image with a first resolution can be generated based on the second image.

[0095] The third image can be generated based on the second image, but with the same resolution as the first image.

[0096] In some alternative implementations, a third image at the first resolution can be generated by downsampling the second image.

[0097] In some alternative implementations, the detection result of the panel support column to be detected can be determined by comparing the consistency of the edge structure between the third image and the first image; alternatively, the final detection result of the panel support column to be detected can be determined by calculating the error information between the third image and the first image.

[0098] It is understandable that since uncertainty information can reflect the reconstruction reliability of the second image, incorporating uncertainty information when determining the detection result allows for differentiated consideration of anomaly determination for second image regions with different levels of reliability. For example, it can increase the confidence level of anomaly determination for regions with high uncertainty (low reconstruction reliability) and decrease the confidence level of anomaly determination for regions with low uncertainty (high reconstruction reliability). Furthermore, the combined determination method of the third image, the first image, and uncertainty information further improves the accuracy and reliability of the detection result, and can avoid false detections or missed detections caused by unreliable local reconstruction of the second image.

[0099] In some optional implementations, a third image is first generated based on the second image. The resolution of the third image is the same as the first resolution. Then, based on the third image and the first image, the detection result of the panel support column to be detected is determined. Specifically, the detection result of the panel support column to be detected can be determined based on the third image, the first image, and uncertainty information as follows: The first step is to determine the error information between the third image and the first image.

[0100] Among them, error information can characterize the differences between the third image and the first image in terms of pixel values, edge structure, texture features, etc.

[0101] In some alternative implementations, the error information can be obtained by calculating the sum of squares of the differences in grayscale values ​​between corresponding pixels in the third image and the first image; or, the error information can be obtained by calculating the cosine similarity between the edge intensity maps of the third image and the first image.

[0102] The second step is to determine the abnormal threshold of the support column of the panel to be tested based on the uncertainty information.

[0103] The anomaly threshold can be used to determine whether there are any anomalies in the support columns of the panel under test. In some cases, the anomaly threshold can be positively correlated with the uncertainty represented by the uncertainty information.

[0104] In some alternative implementations, a higher anomaly threshold can be set for regions with higher uncertainty represented by the uncertainty information, and a lower anomaly threshold can be set for regions with lower uncertainty represented by the uncertainty information. As an example, the anomaly threshold T(u) of the support column of the panel to be detected can be determined based on the uncertainty information ū_ROI using the following formula: T(u) = T0 + α × ū_ROI Formula (1) In formula (1), ū_ROI is the mean / quantile of uncertainty in the ROI (i.e., the panel support column area of ​​the second image); the larger ū_ROI is, the larger the anomaly threshold T(u); T0 represents the basic anomaly threshold; α is the adjustment coefficient used to balance the influence of uncertainty on the anomaly threshold.

[0105] The third step is to determine the detection results of the support column of the panel to be tested based on error information and abnormal threshold.

[0106] Specifically, if the error information indicates an error greater than the abnormal threshold of the corresponding area, then the detection result of the support column of the panel to be tested indicates that there is an abnormality in that area; otherwise, it is judged as normal.

[0107] It is understandable that by determining the anomaly threshold through uncertainty information, the anomaly threshold can be adaptively adjusted according to the reconstruction reliability of the second image. This further improves the accuracy of the detection results and effectively distinguishes between real and false anomalies.

[0108] In some alternative implementations, the detection result of the support column of the panel to be detected can also be determined based on the third image and the first image, as follows: The first step is to determine the area of ​​the first panel support column in the first image.

[0109] The first panel support column area can be an image area in the first image that contains the panel support column to be detected.

[0110] In some alternative implementations, the first panel support pillar region in the first image can be determined using detection and segmentation algorithms. For example, the first panel support pillar region in the first image can be determined using a PS pillar detection / segmentation task head.

[0111] The second step is to determine the second panel support column area in the third image that corresponds to the first panel support column area.

[0112] The second panel support column area can be an image area in the second image that contains the panel support column to be detected.

[0113] In some alternative implementations, the second panel support pillar region in the second image can be determined using detection and segmentation algorithms. For example, the second panel support pillar region in the second image can be determined using a PS pillar detection / segmentation task head.

[0114] In some alternative implementations, the position of the second panel support column region in the second image may correspond to the position of the first panel support column region in the first image.

[0115] The third step is to determine the detection results of the support columns of the panel to be tested based on the first panel support column area and the second panel support column area.

[0116] In some alternative implementations, the differences in features (such as grayscale distribution features and edge sharpness features) between the first panel support column region and the second panel support column region can be compared to determine the detection result of the panel support column to be detected.

[0117] It is understandable that determining the detection result based on the first panel support column area and the second panel support column area allows for a more precise comparison of the features of the first panel support column area and the second panel support column area. This enables a more accurate identification of whether the abnormality of the panel support column to be detected is a real abnormality, avoiding misjudgment caused by differences in the overall background of the image. This further improves the accuracy and reliability of the detection results and reduces the risk of incorrect judgment.

[0118] In some alternative implementations, after determining the detection result of the support column of the panel to be detected based on the third image and the first image, the first image, the second image, the third image, and the detection result can also be output.

[0119] In some optional implementations, the first image, second image, third image, and detection results can be transmitted to a host computer or display device for viewing by staff or further processing by the system. For example, the first image, second image, third image, and detection results including anomaly type and confidence level can be output to the display device; alternatively, the first image, second image, third image, and detection results requiring verification can also be output to the display device.

[0120] It is understandable that by outputting the first image, the second image, the third image, and the detection results, the relevant image data and final judgment conclusions of the detection process can be traced and verified, making it easier for staff to verify the correctness of the detection results. Outputting the detection results can provide a clear basis for subsequent processing, such as triggering the repair or rejection process of the abnormal panel, or reminding staff to review suspicious detection results. In this way, the transparency and reliability of the detection process can be improved.

[0121] It should be noted that, where there is no conflict, the technical features described in different alternative implementations can be included in the same embodiment. For the sake of brevity, they will not be elaborated here.

[0122] Based on the embodiments of this disclosure, a super-resolution model is trained by acquiring a target image at a second resolution using an image acquisition device. Then, the super-resolution model is used to perform super-resolution reconstruction on a low-resolution first image to obtain a high-resolution second image. This can reduce the illusions present in the second image. Based on the second image, panel support columns can be detected, which can reduce the probability of false detection and false negative detection and improve the detection accuracy of panel support columns.

[0123] Figure 2 This is a flowchart illustrating another method for detecting panel support columns provided in an embodiment of this disclosure. Figure 2 As shown, the method specifically includes: Step 201: Obtain the first image of the support column of the panel to be detected.

[0124] In this embodiment, step 201 and Figure 1 Step 101 in the corresponding embodiment is basically the same, and will not be repeated here.

[0125] Step 202: Determine the magnification scale information of the second resolution of the second image relative to the first resolution of the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image.

[0126] In this embodiment, the second resolution may be the resolution of the second image.

[0127] The first resolution can be the resolution of the first image.

[0128] Magnification scale information can represent the magnification factor of the second resolution relative to the first resolution. Magnification scale information can be used in super-resolution models to adapt to different input resolutions (i.e., the resolution of the first image) and the second resolution. As an example, the magnification scale information can be the pixel size of the first resolution divided by the pixel size of the second resolution. For instance, if the first resolution is 3 micrometers per pixel and the second resolution is 2 micrometers per pixel, then the magnification scale information could be 1.5 times.

[0129] Step 203: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image and magnification scale information. The super-resolution model is used to generate the second resolution image, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device.

[0130] In this embodiment, the first image and magnification information can be input into the super-resolution model. The super-resolution model combines the magnification information to adjust the resolution of the first image and generate a second image of the support column of the panel to be detected.

[0131] In some alternative implementations, the super-resolution model can be trained using machine learning algorithms, based on information including the target image, the corresponding low-resolution image, and the magnification scale of the target image relative to the corresponding low-resolution image.

[0132] Step 204: Based on the second image, determine the detection result of the support column of the panel to be detected.

[0133] In this embodiment, step 204 and Figure 1 Step 103 in the corresponding embodiment is basically the same, and will not be repeated here.

[0134] It should be noted that, in addition to the contents described above, this embodiment may also include... Figure 1 The corresponding technical features described in the corresponding embodiments, thereby achieving Figure 1 For details on the technical effectiveness of the detection method for the panel support columns shown, please refer to [link / reference needed]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0135] Based on the embodiments of this disclosure, a second image is generated by a super-resolution model based on a first image and magnification scale information, enabling the super-resolution model to adapt to different conversion requirements from first resolution to second resolution, further improving the accuracy of second image generation and improving the detection accuracy of panel support columns.

[0136] Figure 3 This is a flowchart illustrating another method for detecting panel support columns provided in this embodiment of the disclosure. Figure 3As shown, the method specifically includes: Step 301: Obtain the first image of the support column of the panel to be detected.

[0137] In this embodiment, step 301 and Figure 1 Step 101 in the corresponding embodiment is basically the same, and will not be repeated here.

[0138] Step 302: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected and uncertainty information of the second image are generated based on the first image. The first resolution of the first image is smaller than the second resolution of the second image. The super-resolution model is used to generate the image at the second resolution. The training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device.

[0139] In this embodiment, the uncertainty information of the second image can represent the reliability of the second image generated by the super-resolution model; or, the reliability of each pixel of the second image.

[0140] In some alternative implementations, an uncertainty estimation branch can be embedded in the super-resolution model, and uncertainty information can be learned and output through heteroscedasticity regression or other methods; alternatively, uncertainty information can be calculated based on the pixel error distribution of the reconstructed image.

[0141] Step 303: Based on the second image and uncertainty information, determine the detection result of the support column of the panel to be detected.

[0142] In this embodiment, the detection result of the panel support column to be detected can be determined by combining the structural features of the second image and the reliability of the uncertainty information. As an example, when the uncertainty information represents the reliability of each pixel in the second image, a lower weight (hereinafter referred to as the first weight) can be assigned to the pixel region in the second image where the uncertainty represented by the uncertainty information is greater than a preset threshold, and a higher weight (hereinafter referred to as the second weight, the second weight being greater than the first weight) can be assigned to the pixel region in the second image where the uncertainty represented by the uncertainty information is less than or equal to the preset threshold. Thus, a weighted fusion is performed based on the first weight, the second weight, and the pixel values ​​of each pixel in the second image. The detection result of the panel support column to be detected is determined by the result obtained after the weighted fusion.

[0143] In some optional implementations of this embodiment, the detection result of the support column of the panel to be detected can also be determined based on the second image and uncertainty information in the following manner: The first step is to determine the abnormal threshold of the support column of the panel to be tested based on the uncertainty information.

[0144] The anomaly threshold can be used to determine whether there are any anomalies in the support columns of the panel under test. In some cases, the anomaly threshold can be positively correlated with the uncertainty represented by the uncertainty information.

[0145] In some alternative implementations, a higher anomaly threshold can be set for regions with higher uncertainty represented by the uncertainty information, and a lower anomaly threshold can be set for regions with lower uncertainty represented by the uncertainty information. As an example, the anomaly threshold T(u) of the support column of the panel to be detected can be determined based on the uncertainty information ū_ROI using the following formula: T(u) = T0 + α × ū_ROI Formula (1) In formula (1), ū_ROI is the mean / quantile of uncertainty within the ROI (Region of Interest; i.e., the panel support column area of ​​the second image); the larger ū_ROI is, the larger the anomaly threshold T(u); T0 represents the basic anomaly threshold; α is the adjustment coefficient used to balance the influence of uncertainty on the anomaly threshold.

[0146] The second step is to determine the detection results of the support column of the panel to be detected based on the second image and the anomaly threshold.

[0147] In some alternative implementations, the detection result of the support column of the panel to be detected can be determined based on the second image and the anomaly threshold using the following formula: z=| -μ_h|÷sqrt(σ_h 2 +σ0 2 ) Formula (2) In formula (2), The second image represents the observed value (e.g., the measured height of the PS column); μ_h represents the reference value (e.g., the average height of a normal PS column); σ_h 2 σ0 represents the variance of the observations in the second image. 2 This represents the variance of the reference value.

[0148] Therefore, if z ≥ T(u), the test result indicates that the support column is abnormal; otherwise, the test result indicates that it is normal.

[0149] In some alternative implementations, the feature values ​​of the panel support columns to be detected extracted from the second image can be compared with an anomaly threshold. If the feature value exceeds the threshold, the detection result is determined to be abnormal; if it does not exceed the threshold, the detection result is determined to be normal.

[0150] It is understandable that determining the detection result based on the second image and the adaptively adjusted anomaly threshold can make the anomaly judgment more consistent with the actual reliability of the second image, improve the accuracy of the anomaly judgment, further reduce the risk of false detection / false detection of the support column of the panel to be detected, and improve the reliability of the detection result.

[0151] It should be noted that, in addition to the contents described above, this embodiment may also include... Figure 1 The corresponding technical features described in the corresponding embodiments, thereby achieving Figure 1 For details on the technical effectiveness of the detection method for the panel support columns shown, please refer to [link / reference needed]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0152] Based on the embodiments of this disclosure, by determining the detection result based on the second image and uncertainty information, the risk of false detection or missed detection caused by unreliable pixels in the second image can be reduced, making the judgment of the detection result more targeted, and further improving the accuracy and reliability of the detection result of the support column of the panel to be detected.

[0153] The following describes the embodiments of this disclosure by way of example. However, it should be noted that the following content is only used to understand the technical solutions of the embodiments of this disclosure and does not constitute a limitation on the protection scope of the embodiments of this disclosure.

[0154] In related technologies, panel products often contain regularly arrayed PS pillar structures used for support, gap control, or structural positioning. In AOI (Automated Optical Inspection Image) inspection, the appearance, edge morphology, and local texture of the PS pillars are crucial for anomaly identification, and height anomalies often cause changes in appearance such as reflection intensity distribution, ring shadows, and edge sharpness. However, limited by production line imaging speed, field of view, and hardware costs, lower resolution images (e.g., 5, 4, or 3 micrometers / pixel) are often used in actual inspections, resulting in blurred PS pillar boundaries, insufficient detail, and unstable positioning, affecting the reliability of subsequent height anomaly determination.

[0155] In some super-resolution methods, generative models tend to introduce high-frequency details inconsistent with the input observations (i.e., illusions), which can directly lead to false positives or false negatives in industrial inspection scenarios and make it difficult to provide traceable evidence. Therefore, a technical solution is needed that can significantly improve resolution while strictly constraining realism, and can provide verification evidence and risk classification output during the inference stage.

[0156] This solution can upscale low-resolution (i.e., first-resolution) AOI images (e.g., 5, 4, or 3 micrometers / pixel) to the target resolution (i.e., second-resolution, e.g., 2 micrometers / pixel), while suppressing false details (illusions) caused by generative models in related technologies, thereby improving the measurability, localization accuracy, and reliability of anomaly detection of PS pillars. By constructing a training dataset with the target resolution (i.e., the second resolution) image (i.e., the target image) as the ground truth, and establishing a degradation model consistent with actual imaging, a scale-conditional super-resolution network (i.e., the super-resolution model) is trained to output a high-resolution (i.e., the second resolution) reconstructed image (including the second image) and a pixel-level uncertainty map (i.e., uncertainty information, such as the uncertainty of each pixel). Degradation consistency constraints ensure that the reconstructed image, after degradation back to the original resolution (i.e., the first resolution) (resulting in the third image), is consistent with the input observation (i.e., the fourth image), and high-frequency edges not supported by the input (i.e., the fourth image or the upsampled image) are suppressed by adding edge penalties. A PS column detection / segmentation task head and a height anomaly regression / classification task head are jointly trained. During the inference phase, back-projection verification is performed on suspected anomalous regions. For example, if it is determined that the support column of the panel to be detected is abnormal, a third image is generated based on the second image, and risk classification is output based on the uncertainty (i.e., uncertainty information). This significantly improves resolution while ensuring realism, reducing the risk of false detection / missed detection and incorrect judgment, and is suitable for industrial vision inspection scenarios such as AOI imaging and defect detection in display panels, semiconductors, and related precision manufacturing.

[0157] Overall, such as Figure 4 As shown, this solution includes: 1) Data construction: AOI images acquired at the target resolution (i.e., the second resolution, for example, 2 micrometers / pixel) are used as high-resolution ground truth images (i.e., target images). A degradation model consistent with actual imaging is established for the input resolution (i.e., the first resolution, for example, 3 / 4 / 5 micrometers / pixel). The ground truth images are degraded to generate low-resolution training images (i.e., the fourth image). When real multi-resolution paired acquisition exists, real paired training samples (including the fourth image and the target image) can also be obtained through cross-resolution registration.

[0158] 2) Model structure: Scale-conditional super-resolution network is constructed. The input is a low-resolution image (i.e., the fourth image) and magnification scale information (i.e., magnification scale information). The output is a reconstructed high-resolution image (i.e., the reconstructed image in the inference stage and the second image in the training stage) and a pixel-level uncertainty map. Based on the output, PS column detection / segmentation task head and high anomaly regression / classification task head are constructed.

[0159] 3) Realism Constraint Training: Training is performed using a joint loss function. This joint loss function includes at least: pixel reconstruction loss (to determine the first loss value), degradation consistency loss (to determine the second loss value), and an additional edge penalty term (to determine the third loss value), with optional frequency domain consistency or gradient consistency constraints. The additional edge penalty term penalizes edge energy added to the reconstructed image that is not supported by the input, and a threshold τ can be set (penalization only occurs when the value exceeds τ).

[0160] 4) Reasoning and verification: In the reasoning stage, PS column localization and anomaly detection are performed on the reconstructed image; back projection verification is performed on suspected abnormal areas, the reconstruction result of the area is degraded back to the input resolution (i.e., the first resolution) and compared with the input observation (i.e., the first image), and risk classification is output in combination with the uncertainty map.

[0161] In some embodiments, the training data construction and registration process in this scheme includes: acquiring AOI images at a target resolution of 2 micrometers / pixel as a high-resolution ground truth (i.e., target image) library. Input resolutions of 3 micrometers / pixel, 4 micrometers / pixel, and 5 micrometers / pixel correspond to magnification ratios (i.e., magnification scale information) of 1.5x, 2x, and 2.5x, respectively. In the real-pair acquisition mode, images of the target resolution and the input resolution (i.e., the first resolution) (i.e., the first image) are acquired and registered in the same physical area of ​​the same panel; in the degradation generation mode, a degradation model is established and the parameter distribution is calibrated using a small amount of real low-resolution data to generate training samples. This involves collecting and registering a small number of real fourth image LR (3 / 4 / 5µm / px) and corresponding (at the same location) target image HR (2 µm / px) samples; setting the degradation model D_r = Downsample(Conv(HR,k(θ))) + n(φ), where Downsample() represents the downsampling operation, Conv() represents the convolution operation, θ and φ represent the model parameters, k(θ) represents the function of θ, and n(φ) represents the function of φ; optimizing θ and φ to minimize the loss (using L1+SSIM or frequency domain loss) between the third image D_r (HR) obtained after processing the reconstructed image by the degradation model and the real fourth image LR; fitting parameter distributions and versioning them for different production lines / formulas / states; training / inference loading the corresponding degradation model D_r according to the formula to calibrate the parameter distribution.

[0162] In some embodiments, the construction and uncertainty output process of the scale-conditional super-resolution network in this scheme includes: constructing a scale-conditional super-resolution network (i.e., a super-resolution model) F, with the input being a low-resolution image (i.e., the first image) I_LR and a magnification scale (i.e., magnification scale information) s, and the output being a reconstructed image. _HR and pixel-level uncertainty map (i.e., magnified scale information) σ. The uncertainty map σ is learned using heteroscedasticity regression and is used for risk grading and threshold adaptation during the inference stage.

[0163] In some embodiments, the realism constraint training loss function in this scheme can adopt a joint loss function L = λ1×L_rec + λ2×L_dc + λ3×L_hall + λ4×L_grad÷freq + λ5×L_task. Wherein, L_rec represents the first loss value mentioned above, used to constrain the consistency between the reconstructed image and the target image; L_dc represents the second loss value mentioned above, used to constrain the consistency between the fourth image and the third image; L_hall represents the third loss value mentioned above, used to penalize newly added edge energy; L_grad represents the gradient / edge loss value between the reconstructed image and the fourth image, used to constrain the gradient / edge consistency between the reconstructed image and the fourth image to suppress texture drift and maintain structural boundary stability; freq represents the loss value between the reconstructed image and the fourth image in the FFT (Fast Fourier Transform) amplitude spectrum / bandpass energy, used to suppress texture drift; L_task represents the PS column detection / segmentation and height anomaly task loss value; λ1, λ2, λ3, λ4, and λ5 are preset weights.

[0164] In some embodiments, the PS column identification and height anomaly detection process in this scheme includes: performing PS column detection / segmentation on the reconstructed image to obtain the ROI of each PS column; performing height regression and anomaly classification when ground truth heights exist, performing appearance proxy anomaly classification and gradually supplementing the supervision signal when ground truth heights are lacking; and reducing the risk of misjudgment by combining the uncertainty graph σ.

[0165] In some embodiments, the back-projection verification and evidence output process in this scheme includes: reconstructing images for suspected anomalous ROIs. The third image is obtained by degrading the HR image back to the input resolution using the degradation model D_r. `_LR_back` performs a consistency comparison with the input observation (i.e., the fourth image). When the error exceeds a threshold, the output is downgraded (i.e., when the verification fails or the uncertainty is too high, instead of directly giving "definite anomaly", the result is downgraded from "anomaly (high confidence)" to "suspicious / requires verification", or the anomaly message is retained but the confidence level is significantly reduced and a re-judgment / re-scanning process is triggered) or a verification is triggered. The system outputs a comparison of three images (first image, second image, and third image) and the verification error (i.e., the detection result), forming a closed loop of traceable evidence.

[0166] Specifically, this plan includes: (1) Obtain the high-resolution image of the target resolution (i.e. the target image) as the ground image, and establish a degradation model for the low-resolution input resolution to be improved (i.e. the first resolution), and degrade the ground image to generate a low-resolution image for training (i.e. the fourth image). (2) Construct a scale-conditional super-resolution network, using the low-resolution image and scale information (i.e. magnified scale information) used for training as input, and output the reconstructed high-resolution image (i.e. reconstructed image) and pixel-level uncertainty map (i.e. uncertainty information). (3) The scale-conditional super-resolution network is trained using a joint loss function that includes pixel reconstruction loss, degradation consistency loss, and a newly added edge penalty term. See [link to relevant documentation]. Figure 5 The degradation consistency loss is used to constrain the reconstructed high-resolution image to be consistent with the corresponding input low-resolution image after degradation by the degradation model; see [link to relevant documentation]. Figure 6 A new edge penalty term is added to suppress high-frequency edge structures in the reconstructed high-resolution image that are not supported by the input low-resolution image; (4) Based on the output of the scale-conditional super-resolution network, jointly train the PS column detection / segmentation task head and the height anomaly regression / classification task head (see Figure 7 ); (5) During the reasoning phase, back projection verification is performed on the PS column regions suspected of being highly anomaly (see Figure 8 The process involves reconstructing the image corresponding to the region, degrading it back to the input resolution using a degradation model, and comparing its consistency with the input observations. This is combined with a pixel-level uncertainty map to output anomaly conclusions and risk levels (i.e., detection results). For example, multi-source threshold gating can be used: the anomaly head outputs anomaly probability p or height deviation Δh (deviation relative to the true value), combined with detection / segmentation confidence c, uncertainty u, and backprojection consistency error e. Initial screening can be performed using anomaly probability p > probability threshold T_p or absolute value of height deviation |Δh| > deviation threshold T_h; further screening is performed using backprojection consistency error e < error threshold T_e or uncertainty u < uncertainty threshold T_u, otherwise it is downgraded to suspicious / re-verified. Furthermore, grading can be based on p, Δh, e, u, and c. For example, Level 1 (Normal): p≤T1 and e≤E1 and u≤U1; Level 2 (Suspicious): p>T1 and e≤E1 and u≤U2; Level 3 (High Risk): p>T2 and e≤E2 and u≤U2; Level 4 (Requires Review / Downgrade): e>E2 or u>U3 (any trigger). Here, T1 and T2 represent different probability thresholds, and T1<T2; E1 and E2 represent different error thresholds, and E1<E2; U1, U2, and U3 represent different uncertainty thresholds, and U1<U2<U3.

[0167] In some embodiments, the degradation model includes at least two of point spread function convolution, downsampling operator, and noise injection operator, wherein the noise injection operator is used to simulate the actual imaging noise distribution.

[0168] In some embodiments, the scale information includes magnification information, for example, the magnification includes at least one of 1.5x, 2x, and 2.5x, to upscale an input of 3 micrometers / pixel, 4 micrometers / pixel, or 5 micrometers / pixel to 2 micrometers / pixel.

[0169] In some embodiments, the pixel-level uncertainty map is learned via heteroscedastic regression and employs a negative log-likelihood loss L_i = (y_i - _i) 2 ÷(2σ_i 2 ) + 0.5×ln(σ_i 2 The reconstruction error is weighted to correlate the uncertainty with the reconstruction error. Here, i is used to identify the pixel, L_i represents the negative log-likelihood loss value of pixel i, and y_i represents the pixel value of pixel i in the target image. _i represents the pixel value of pixel i in the reconstructed image, and σ_i represents the variance of pixel i predicted by the model. In some embodiments, the added edge penalty term includes: calculating an edge intensity map for the reconstructed image, calculating an edge intensity map after upsampling the input low-resolution image, and applying a thresholding penalty to the portion of edge energy added to the reconstructed image relative to the upsampled input image.

[0170] In some embodiments, the PS column detection / segmentation task head outputs a segmentation mask, a center point heatmap, or a combination of both for the PS column to determine the region or center location of each PS column.

[0171] In some embodiments, the high anomaly regression / classification task head can output a high regression value and anomaly probability when a high ground truth value exists; when a high ground truth value does not exist, it can output anomaly probability based on appearance proxy features (such as color, texture, shape, etc.).

[0172] In some embodiments, backprojection verification includes: calculating an error metric between the image degraded back to the input resolution and the input observation. When the error metric exceeds a threshold, the abnormal conclusion is downgraded to require review or the confidence level is reduced.

[0173] In some embodiments, the joint training loss function further includes frequency domain consistency constraints or gradient consistency constraints to suppress texture drift and maintain structural boundary stability.

[0174] In some embodiments, the training dataset is generated by collecting and registering the same region of the same panel at different resolutions, or by degrading the ground truth image at the target resolution, and the corresponding training sample is removed when the registration residual (representing the difference between the reconstructed image and the fourth image after degradation back to the third image) exceeds a threshold.

[0175] In some embodiments, the inference stage employs an overlapping tile method to perform super-resolution reconstruction of large-format images and merges the tile boundary regions to reduce the impact of the stitching seams on the edges of the PS columns.

[0176] In some embodiments, the PS column position, anomaly type, confidence level, and detection results can be output to the host computer or manufacturing execution system.

[0177] In some embodiments, degradation model parameters can be configured and versioned according to production line, recipe, or equipment status to adapt to different imaging conditions and ensure consistency.

[0178] In some embodiments, the uncertainty graph is used for anomaly decision threshold adaptation: when the uncertainty is greater than the threshold, the abnormal output is marked as suspicious and a review process is triggered.

[0179] In some embodiments, the super-resolution network is a Transformer structure, a convolutional residual structure, or a combination of both, and includes scale embedding to achieve unified modeling of multiple-scale inputs.

[0180] In some embodiments, the PS column detection / segmentation task head and the height anomaly task head share some feature backbones with the super-resolution network to utilize structural information to constrain the super-resolution results and improve task performance.

[0181] In some embodiments, backprojection verification is performed at the ROI level and outputs comparative evidence, including the input first image LR (Low-Resolution), the reconstructed second image SR (Super-Resolution), and the third image degraded back to LR, for three-image comparison.

[0182] In some embodiments, the training process of the super-resolution model, the PS column detection / segmentation task head, and the height anomaly regression / classification task head includes: first performing fidelity super-resolution pre-training, then performing fine-tuning for adaptation in the real low-resolution domain, and finally performing multi-task joint training to converge PS column detection and height anomaly determination.

[0183] It should be noted that, in addition to the contents described above, this embodiment may also include the technical features described in the above embodiments, thereby achieving the technical effect of the panel support column detection method shown above. Please refer to the above description for details. For the sake of brevity, it will not be elaborated here.

[0184] This solution significantly improves the resolution of images used for detection, enhancing the sharpness and localization stability of PS column edges. It suppresses illusions through degradation consistency and added edge penalties, ensuring interpretability. Pixel-level uncertainty is output and verified through backprojection, forming a closed-loop evidence system. Multi-task joint training improves the reliability of high-anomaly detection. It supports multiple magnification inputs (3 / 4 / 5 micrometers / pixels) to uniformly upscale to the target resolution. Furthermore, this solution can be deployed on production line inference servers or edge computing devices, supporting large-format image block reconstruction and fusion. Degradation model parameters can be versioned to adapt to different imaging conditions. While improving resolution, it provides consistency verification and uncertainty risk grading, making it suitable for panel PS column structure recognition and high-anomaly screening, and can be extended to other regular structure industrial vision inspection tasks.

[0185] Figure 9 This is a schematic diagram of a panel support column detection device provided in an embodiment of the present disclosure. The panel support column detection device includes: The acquisition unit 401 is configured to: acquire a first image of the support column of the panel to be detected; The generation unit 402 is configured to: generate a second image of the panel support column to be detected based on the first image using a pre-trained super-resolution model, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. The determining unit 403 is configured to: determine the detection result of the support column of the panel to be detected based on the second image.

[0186] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Determine the magnification scale information of the second resolution of the second image relative to the first resolution of the first image; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image and magnification scale information.

[0187] In some possible implementations, a second image of the panel support column to be detected is generated based on the first image using a pre-trained super-resolution model, including: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, along with uncertainty information of the second image; Based on the second image, the detection results of the support column of the panel to be detected are determined, including: Based on the second image and uncertainty information, the detection result of the support column of the panel to be detected is determined.

[0188] In some possible implementations, the detection result of the support column of the panel to be detected is determined based on the second image and uncertainty information, including: Based on uncertainty information, the abnormal threshold of the support column of the panel to be detected is determined; Based on the second image and the anomaly threshold, the detection results of the support column of the panel to be detected are determined.

[0189] In some possible implementations, the super-resolution model is trained as follows: Acquire a fourth image, and a target image acquired by an image acquisition device at a second resolution, wherein the resolution of the fourth image is lower than the second resolution; A super-resolution model is trained based on the fourth image and the target image.

[0190] In some possible implementations, a super-resolution model is trained based on the fourth image and the target image, including: The fourth image is input into the initial super-resolution model to obtain a reconstructed image at the second resolution. Based on the reconstructed image and the target image, determine the first loss value; A third image corresponding to the reconstructed image is generated using a pre-trained degradation model, wherein the degradation model is used to generate a third image with a resolution lower than that of the reconstructed image. Based on the fourth and third images, determine the second loss value; Upsample the fourth image to obtain an upsampled image with a second resolution; The third loss value is determined based on the edge information of the upsampled image and the edge information of the reconstructed image; Based on the first loss value, the second loss value, and the third loss value, the model parameters of the initial super-resolution model are adjusted to obtain the trained super-resolution model.

[0191] In some possible implementations, a first loss value is determined based on the reconstructed image and the target image, including: Based on the reconstructed image and the target image, determine the error information of the reconstructed image relative to the target image; Based on the error information and the uncertainty information of the reconstructed image, the first loss value is determined.

[0192] The panel support column detection device provided in this embodiment can execute the corresponding steps of the panel support column detection methods described above, thereby achieving the technical effects of the panel support column detection methods described above. The panel support column detection device and the panel support column detection methods can refer to and cite each other in terms of specific implementation and technical effects. For the sake of brevity, they will not be elaborated here.

[0193] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Figure 10 The illustrated electronic device 500 includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to implement communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 10 The general designated all buses as Bus System 505.

[0194] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0195] It is understood that the memory 502 in this embodiment can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0196] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.

[0197] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this disclosure embodiment can be included in the application program 5022.

[0198] In this embodiment, by calling the program or instructions stored in memory 502, specifically the program or instructions stored in application program 5022, processor 501 executes the method steps provided in each method embodiment, including, for example: Acquire the first image of the support column of the panel to be inspected; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. Based on the second image, the detection results of the support column of the panel to be detected are determined.

[0199] The methods disclosed in the above embodiments of this disclosure can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 502. Processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0200] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described above, or combinations thereof.

[0201] For software implementation, the techniques described herein can be implemented by units that perform the functions described above. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0202] The electronic device provided in this embodiment may be as follows: Figure 10 The electronic device shown can perform all the steps of the detection method for each panel support column described above, thereby achieving the technical effect of the detection method for each panel support column described above. For details, please refer to the relevant descriptions above. For the sake of brevity, it will not be elaborated here.

[0203] This disclosure also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory.

[0204] When one or more programs in the storage medium can be executed by one or more processors to implement the above-described detection method for panel support columns executed on the electronic device side.

[0205] The processor described above is used to execute a detection program for panel support pillars stored in memory to implement the following steps of a panel support pillar detection method executed on the electronic device side: Acquire the first image of the support column of the panel to be inspected; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. Based on the second image, the detection results of the support column of the panel to be detected are determined.

[0206] Furthermore, the computer program product provided in this disclosure embodiment may include computer-readable code that, when executed on a device, causes a processor in the device to implement the steps of the panel support column detection method executed on the electronic device side: Acquire the first image of the support column of the panel to be inspected; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device. Based on the second image, the detection results of the support column of the panel to be detected are determined.

[0207] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0208] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0209] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0210] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for detecting panel support columns, characterized in that, The method includes: Acquire the first image of the support column of the panel to be inspected; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes the target image acquired at the second resolution via an image acquisition device; Based on the second image, the detection result of the support column of the panel to be detected is determined.

2. The method according to claim 1, characterized in that, The step of generating a second image of the panel support column to be detected based on the first image using a pre-trained super-resolution model includes: Determine the magnification scale information of the second resolution of the second image relative to the first resolution of the first image; Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected is generated based on the first image and the magnification scale information.

3. The method according to claim 1, characterized in that, The step of generating a second image of the panel support column to be detected based on the first image using a pre-trained super-resolution model includes: Using a pre-trained super-resolution model, a second image of the support column of the panel to be detected and uncertainty information of the second image are generated based on the first image; and The step of determining the detection result of the support column of the panel to be detected based on the second image includes: Based on the second image and the uncertainty information, the detection result of the support column of the panel to be detected is determined.

4. The method according to claim 3, characterized in that, The step of determining the detection result of the support column of the panel to be detected based on the second image and the uncertainty information includes: Based on the uncertainty information, the abnormal threshold of the support column of the panel to be detected is determined; Based on the second image and the anomaly threshold, the detection result of the support column of the panel to be detected is determined.

5. The method according to any one of claims 1-4, characterized in that, The super-resolution model was trained in the following manner: Acquire a fourth image, and a target image acquired by an image acquisition device at the second resolution, wherein the resolution of the fourth image is lower than the second resolution; The super-resolution model is trained based on the fourth image and the target image.

6. The method according to claim 5, characterized in that, The process of training the super-resolution model based on the fourth image and the target image includes: The fourth image is input into the initial super-resolution model to obtain the reconstructed image at the second resolution; Based on the reconstructed image and the target image, a first loss value is determined; A third image corresponding to the reconstructed image is generated using a pre-trained degradation model, wherein the degradation model is used to generate a third image with a resolution lower than that of the reconstructed image; Based on the fourth image and the third image, a second loss value is determined; The fourth image is upsampled to obtain an upsampled image with the second resolution; Based on the edge information of the upsampled image and the edge information of the reconstructed image, a third loss value is determined; Based on the first loss value, the second loss value, and the third loss value, the model parameters of the initial super-resolution model are adjusted to obtain the trained super-resolution model.

7. The method according to claim 6, characterized in that, Determining the first loss value based on the reconstructed image and the target image includes: Based on the reconstructed image and the target image, error information of the reconstructed image relative to the target image is determined; Based on the error information and the uncertainty information of the reconstructed image, a first loss value is determined.

8. A detection device for panel support columns, characterized in that, The device includes: The acquisition unit is configured to: acquire a first image of the support column of the panel to be detected; The generation unit is configured to: generate a second image of the support column of the panel to be detected based on the first image using a pre-trained super-resolution model, wherein the first resolution of the first image is smaller than the second resolution of the second image, the super-resolution model is used to generate the image at the second resolution, and the training data of the super-resolution model includes a target image acquired at the second resolution via an image acquisition device; The determining unit is configured to: determine the detection result of the support column of the panel to be detected based on the second image.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory, wherein when the computer program is executed, it implements the method described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images