Glass detection method and electronic equipment

Abstract

The invention discloses a glass detection method and electronic equipment. The method comprises the following steps: acquiring an optical image and a vector image of to-be-detected glass; wherein the vector image of the to-be-detected glass indicates a standard image of the to-be-detected glass; and inputting the optical image and the vector image of the to-be-detected glass into a target model to obtain a defect detection result corresponding to the to-be-detected glass. On the basis, automatic detection of the defects of the to-be-detected glass is realized, and the detection efficiency of the glass defects is improved. And the accuracy of defect detection is ensured. In addition, missing detection can be avoided.

Description

Technical Field

[0001] This application relates to the field of glass testing technology, and in particular to a glass testing method and electronic equipment. Background Technology

[0002] Defects may occur during the glass manufacturing process. These can include uneven surfaces or incomplete patterns. Therefore, to ensure glass quality, it is necessary to inspect the glass for defects.

[0003] Currently, defects in glass are typically detected manually. However, manual defect detection is inefficient. Summary of the Invention

[0004] This application provides a glass inspection method and electronic device to improve the efficiency of glass defect detection and ensure the accuracy of defect detection.

[0005] On one hand, embodiments of this application provide a glass inspection method applied to electronic devices, the method comprising:

[0006] Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested;

[0007] The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

[0008] In some embodiments, inputting the optical image and vector image of the glass to be inspected into the target model to obtain the defect detection result corresponding to the glass to be inspected includes:

[0009] The vector image of the glass to be tested is simulated to obtain the simulated image features of the glass to be tested;

[0010] Polarization features are extracted from the optical image of the glass to be tested to obtain the polarization image features of the glass to be tested;

[0011] The simulated image features and polarization image features of the glass to be tested are input into the target model to obtain the defect detection results.

[0012] In some embodiments, inputting the optical image and vector image of the glass to be inspected into the target model to obtain the defect detection result corresponding to the glass to be inspected includes:

[0013] The optical image and vector image of the glass to be tested are input into the target model;

[0014] The vector image of the glass to be tested is simulated using the target model to obtain the simulated image features of the glass to be tested, and the polarization features of the optical image of the glass to be tested are extracted to obtain the polarization image features of the glass to be tested.

[0015] The simulated image features and polarization image features of the glass to be tested are processed by the target model to obtain the defect detection results corresponding to the glass to be tested.

[0016] In some embodiments, the process of simulating the vector image of the glass to be detected includes:

[0017] The vector image of the glass to be tested is simulated based on the glass medium optical transmission model; wherein the glass medium optical transmission model is determined based on the point spread function and the transmittance matrix.

[0018] In some embodiments, the process of determining the defect detection result described above may include:

[0019] Using the target model, the polarization image features and the color image of the glass to be tested are fused to obtain the fused features;

[0020] The defect detection result is determined using the target model, based on the fused features and the simulated image features.

[0021] In some embodiments, the process of determining the target model may include:

[0022] Acquire optical sample images and their corresponding vector sample images;

[0023] The initial model is trained based on the optical sample image and its corresponding vector sample image to obtain the target model.

[0024] In some embodiments, the process of training the initial model based on the optical sample image and its corresponding vector sample image to obtain the target model may include:

[0025] The initial model is trained based on the optical sample images and their corresponding vector sample images;

[0026] Calculate the multi-constraint joint loss function corresponding to the trained model; wherein, the multi-constraint joint loss function is obtained by weighting the curvature continuity loss function and the preset loss function;

[0027] If the multi-constraint joint loss function is greater than the preset function value, return to the step of training the initial model based on the optical sample image and its corresponding vector sample image;

[0028] If the multi-constraint joint loss function is less than or equal to the preset function value, the trained model is used as the target model.

[0029] In some embodiments, the curvature continuity loss function is determined based on the partial derivatives of pixels in the defect region corresponding to the optical sample image.

[0030] In some embodiments, the image features corresponding to the optical sample image are obtained by fusing the polarization image features of the optical sample image and the color image corresponding to the optical sample image;

[0031] The polarization image features of the optical sample image are obtained by extracting polarization features from the optical sample image.

[0032] The image features corresponding to the vector sample image are obtained by simulating the vector sample image.

[0033] On the other hand, embodiments of this application also provide a glass inspection device, the device including an image acquisition module and a defect detection module;

[0034] The image acquisition module is used to acquire optical images and vector images of the glass to be inspected; wherein the vector image of the glass to be inspected indicates the standard image of the glass to be inspected.

[0035] The defect detection module is used to input the optical image and vector image of the glass to be tested into the target model to obtain the defect detection result corresponding to the glass to be tested.

[0036] On the other hand, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0037] Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested;

[0038] The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

[0039] On the other hand, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the glass detection methods provided in embodiments of this application.

[0040] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program or instructions thereon, including a computer program or instructions that, when executed by a processor, implement the steps in any of the glass detection methods provided in embodiments of this application.

[0041] The glass inspection method provided in this application acquires optical and vector images of the glass to be inspected when defects need to be detected. The vector image indicates a standard image of the glass, i.e., a standard pattern of the glass. Then, the optical and vector images are used as parameters of a target model, which is run to perform comprehensive processing on the optical and vector images. This allows for the identification of defects in the glass using different types of images, resulting in the corresponding defect detection results. This achieves automatic defect detection, improving the efficiency of glass defect detection and ensuring accuracy. Furthermore, it avoids missed detections. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1-2 This is a schematic diagram of a glass defect scenario provided in an embodiment of this application;

[0044] Figure 3-4 This is a flowchart illustrating an anomaly detection method provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the structure of an anomaly detection device provided in the embodiments of this application;

[0046] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] In the following description, specific embodiments of the invention will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the invention described above are not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.

[0049] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. The various components, modules, engines, and services described herein can be considered as implementations on the computing system. While the apparatus and methods described herein are preferably implemented in software, they can also be implemented in hardware, both of which are within the scope of this invention.

[0050] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when an element is “connected” or “coupled” to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein may include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0051] Glass (such as industrial glass) may have defects such as uneven surface and incomplete patterns during the production process. For example, compared to... Figure 1 Pattern 11 on glass 10, Figure 2 The pattern 21 on the glass 20 is incomplete. Therefore, it is necessary to inspect the glass for defects to determine whether the glass quality meets the requirements.

[0052] Based on this, this application proposes a glass inspection method, specifically a glass defect detection method based on an improved Open-CD framework. By combining the optical and vector characteristics of glass, accurate detection of defects in transparent media is achieved, ensuring the accuracy of defect detection and reducing the probability of false detections and missed detections.

[0053] The following will combine Figure 3 The implementation process of the above glass testing method is described in detail. This method can be executed by electronic equipment. For example... Figure 3 As shown, the method specifically includes S301-S302.

[0054] S301. The electronic device acquires an optical image and a vector image of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested.

[0055] The standard image of the glass to be tested can represent the standard style, i.e., the correct style, of the glass to be tested. Optionally, the standard image of the glass to be tested can be a design drawing of the glass to be tested.

[0056] For example, the standard image of the glass to be tested can be a scalable vector graphic (SVG).

[0057] The optical image of the glass to be tested mentioned above refers to the optical image obtained by taking a picture of the glass to be tested with a camera. It may include the polarization image of the glass to be tested, that is, the polarization direction resolution image.

[0058] Optionally, the polarization image of the glass to be inspected can be acquired by a line scan camera. Specifically, the polarization image can be obtained using a polarization filter of the line scan camera.

[0059] S302. The electronic device inputs the optical image and vector image of the glass to be inspected into the target model to obtain the defect detection result corresponding to the glass to be inspected.

[0060] In this embodiment, after obtaining the optical image and vector image of the glass to be tested, the optical image and vector image are used as input parameters of the target model. The target model is then run, and it processes the optical image and vector image to identify whether there are defects in the glass to be tested, thus obtaining the corresponding defect detection result. In simple terms, the target model compares and analyzes the optical image and vector image of the glass to be tested to determine whether there are defects in the optical image, thereby determining whether there are defects in the glass to be tested and obtaining the corresponding defect detection result.

[0061] The defect detection result for the glass under test indicates whether the glass under test has a defect. For example, when the defect detection result for the glass under test indicates that the glass under test has a defect, the defect detection result for the glass under test may also include the type of defect, such as incomplete glass pattern or incomplete glass itself.

[0062] In some embodiments, the target model described above can be obtained by training an initial model using sample data. For example, the sample data may include optical sample images and vector sample images. An electronic device can acquire the optical sample images and their corresponding vector sample images. Then, the electronic device can use the optical sample images and their corresponding vector sample images to train the initial model to obtain the target model.

[0063] The optical sample image can be an optical sample image of the sample glass. Correspondingly, the vector image corresponding to the optical sample image can also be a vector image of the sample glass. Alternatively, the vector image corresponding to the optical sample image can also be a vector template obtained from a vector template library.

[0064] Optionally, the sample glass may include one or more specifications (or types) of sample glass.

[0065] In some embodiments, the above training process may include: the electronic device training an initial model based on optical sample images and their corresponding vector sample images. Then, the electronic device calculates the multi-constraint joint loss function corresponding to the trained model; wherein the multi-constraint joint loss function is obtained by weighting the curvature continuity loss function and a preset loss function.

[0066] Then, the electronic device determines whether the multi-constraint joint loss function (i.e., the value of the multi-constraint joint loss function) is less than or equal to the preset function value, in order to determine whether the accuracy of the trained model meets the requirements, and thus determine whether to continue training.

[0067] If the joint loss function with multiple constraints exceeds the preset value, it indicates that the accuracy of the trained model is low. Therefore, the electronic device can continue training the trained model, returning to the step of training the initial model based on the optical sample images and their corresponding vector sample images, until the accuracy meets the requirements. Furthermore, the electronic device can adjust the parameters of the trained model during continued training.

[0068] If the multi-constraint joint loss function is less than or equal to the preset function value, it indicates that the accuracy of the trained model is high and meets the requirements. Therefore, the electronic device does not need to continue training and can use the trained model as the target model for inference.

[0069] Optionally, the curvature continuity loss function in the above-mentioned multi-constraint joint loss function can be determined based on the partial derivatives of the pixels in the defect region corresponding to the optical sample image.

[0070] For example, the curvature continuity loss function (i.e., the value of the curvature continuity constraint function) can be determined using Equation 1. Wherein, Equation 1 is...

[0071]

[0072] L curve This represents the curvature continuity loss function. D represents the defect region corresponding to the optical sample image. p represents a pixel within this defect region. x and y represent the coordinates of the pixel. M() represents the defect mask, whose value is either 0 or 1. For example, when pixel p is a defect point, the defect mask value for pixel p can be 1. When pixel p is not a defect point, the defect mask value for pixel p can be 0.

[0073] In this embodiment, morphological constraint optimization is achieved through the curvature continuity loss function, which can filter out false defect regions in the defect region, ensure the accuracy of defect identification, and reduce the probability of false detection by the model.

[0074] Among them, optical sample images are similar to optical images, and vector sample images are similar to vector images, and their corresponding descriptions can be referenced from each other.

[0075] In some embodiments, the aforementioned preset loss function can be a conventional loss function, i.e., a general loss function. Taking the preset loss function as including the Dice loss function and the Pol loss function as an example, the aforementioned multi-constraint joint loss function can be obtained by weighting according to Formula 2.

[0076] Formula 2 is,

[0077]

[0078] L total This represents the joint loss function with multiple constraints. Dice This represents the Dice loss function (the value of the Dice loss function). L curve L represents the curvature continuity loss function. Pol This represents the Pol loss function (i.e., the value of the Pol loss function).

[0079] λ1, λ2, and λ3 are the weights corresponding to the loss function. The initial values ​​of λ1, λ2, and λ3 can be preset. Furthermore, λ1, λ2, and λ3 can be dynamically adjusted during training.

[0080] Optionally, the aforementioned multi-constraint joint loss function can be determined by the dual-task joint optimization model in the initial module, which achieves physically interpretable defect segmentation through curvature continuity constraints, i.e., the curvature continuity loss function.

[0081] It should be understood that the core objective of the aforementioned POL (Policy Optimization Loss) function is to maximize the expected cumulative reward by optimizing the policy. The Dice loss function optimizes the model by measuring the overlap between the predicted results and the true labels.

[0082] Optionally, the aforementioned defect region is determined based on a defect probability map. This defect probability map is determined by performing multi-scale difference calculations on the image features corresponding to the optical sample image and the image features corresponding to the vector sample image. This allows for comparison and analysis of the differences between the image features corresponding to the optical sample image and the image features corresponding to the vector sample image at different scales, thereby determining whether the optical sample image has defects, i.e., whether the sample glass has defects, and obtaining the corresponding defect probability map.

[0083] Optionally, the defect probability map can indicate the defect probability of each pixel in the optical sample image. For each pixel in the defect probability map, if the defect probability of the pixel is greater than or equal to a preset probability threshold, it indicates that the pixel has a high probability of being a defect. Therefore, the electronic device (or alternatively described as the initial model) can use this pixel as a pixel in the defect region.

[0084] If the probability of a pixel being defective is less than a preset probability threshold, it indicates that the probability of that pixel being defective is low. Therefore, the electronic device can determine that the pixel does not belong to the defective area.

[0085] Optionally, the aforementioned preset probability threshold is a dynamic threshold. For example, Threshold = μ + β * σ. Here, Threshold represents the dynamic threshold. μ represents the mean of the defect probability map. σ represents the standard deviation of the defect probability map. β represents a preset coefficient, which varies depending on the glass specifications (or alternatively, the material).

[0086] Furthermore, during the multi-scale differential calculation of the defect probability map, the electronic device can be constrained according to illumination invariance constraints to ensure the accuracy of the defect probability map. The constraints included in the illumination invariance constraints can be set according to requirements, and this application does not impose any restrictions on them.

[0087] Optionally, the aforementioned defect probability map can be obtained by multi-scale difference calculation of the image features and nonlinear mapping matrix corresponding to the optical sample image, using an electronic device (or alternatively described as an initial model). The nonlinear mapping matrix is ​​obtained by spatially mapping vector image features to optical image features.

[0088] For example, the aforementioned nonlinear mapping matrix can be determined by the dual-stream feature alignment module in the initial model. This dual-stream feature alignment module can establish a nonlinear mapping matrix from the vector image space to the optical image space to achieve the fusion of heterogeneous data. Furthermore, establishing the nonlinear mapping matrix requires the use of weights, which are dynamically assigned based on polarization confidence during the initial model's training.

[0089] Optionally, the aforementioned optical image features can be polarization image features of the optical sample image. Specifically, the description of polarization image features can be found in the relevant description below. Additionally, the description of vector image features can also be found in the relevant description below, and will not be described in detail here.

[0090] In some embodiments, the image features corresponding to the aforementioned optical sample image can be obtained by fusing the polarization image features of the optical sample image and the color image corresponding to the optical sample image. Accordingly, the image features corresponding to the optical sample can include the fused features.

[0091] The image features corresponding to the optical sample image can be obtained by fusing them using the polarization attention fusion module in the initial model. For example, the polarization attention fusion module can first calculate the channel weighting α. Then, based on the channel weighting α, the polarization attention fusion module can perform feature fusion on the polarization image features of the optical sample image and the corresponding color image to achieve polarization attention fusion.

[0092] Optionally, the color image corresponding to the optical sample image can be a color image of the sample glass corresponding to the optical sample image. This color image can be an RGB (red, green, blue) image.

[0093] Optionally, α can be determined using Formula 3. Formula 3 is: α = σ(Conv) 1×1 ([F RGB ||F Pol σ is the sigmoid function, and || represents channel concatenation. F Pol This represents the polarization image characteristics of an optical sample image. F RGB This represents the color image corresponding to the optical sample image. Conv represents convolution. Alternatively, F... RGBIt can represent the features of the color image corresponding to the optical sample image, and these features can be obtained by feature extraction from the color image. Optionally, the polarization image features here can also refer to the polarization image.

[0094] Optionally, the polarization attention fusion module can perform feature fusion using Equation 4. Wherein, Equation 4 is F... fusion =α☉F RGB +(1-α)☉F Pol F fusion It can be the fused features, that is, the image features corresponding to the optical sample images.

[0095] In this embodiment, the polarization attention fusion module can dynamically fuse color image and polarization image features using learnable parameters (such as channel weighting) to suppress reflective interference from transparent materials. This allows the target model to adapt to reflective interference from the glass surface, thereby reducing the probability of false defects caused by glass reflection and refraction.

[0096] It should be noted that the above method of determining the image features corresponding to the optical sample image based on the color image and the optical sample image is only one possible implementation. The image features corresponding to the optical sample image can also be determined in other ways. For example, electronic devices can use the polarization image features of the optical sample image as the image features corresponding to the optical sample.

[0097] In some embodiments, the polarization image features of the aforementioned optical sample image are obtained by extracting polarization features from the optical sample image. For example, a polarization feature extraction network can parse the polarization direction features of the optical sample image, i.e., the polarization sample image, to obtain the polarization image features.

[0098] Optionally, the polarization feature extraction network can also perform frequency domain filtering on the polarization sample image to improve the quality of the polarization sample image, thereby improving the reliability of defect identification.

[0099] Optionally, the polarization feature extraction network described above can be part of the initial model or be independent; this application does not limit it.

[0100] In this embodiment, the polarization feature extraction network can extract optical features related to the glass material using polarization filter data from a line scan camera.

[0101] In some embodiments, the image features corresponding to the vector sample image are obtained by simulating the vector sample image. The physical optics correction layer can simulate the vector sample image corresponding to the optical sample image based on the glass medium light transmission model to obtain the image features (or simulated image features, simulated image) corresponding to the vector sample image, thereby simulating the shooting situation of the sample glass and realizing the adaptive calibration of the physically driven template. The glass medium light transmission model is determined based on the point spread function and the transmittance matrix.

[0102] For example, the above-mentioned glass-medium optical transmission model can be Among them, I sim This represents the image features corresponding to the vector sample image. T represents the transmittance matrix. I svg Represents a vector image. PSF represents the point spread function. N speckle This represents speckle noise. Since some noise occurs when photographing glass, speckle noise can be superimposed to simulate the noise and ensure the accuracy of the simulation.

[0103] Optionally, the elements in the transmittance matrix above represent the transmittance of the corresponding pixel in the vector sample image. For example, T ij This represents the transmittance of the pixel in the i-th row and j-th column.

[0104] Optionally, the aforementioned physical optics correction layer can be part of a physics-driven model generator, which may further include a CycleGAN network for unsupervised domain adaptation. The physics-driven model generator performs unsupervised domain adaptation with real production line images, such as the aforementioned vector sample images; this is known as unsupervised adaptation. In image simulation tasks, unsupervised domain adaptation (UDA) improves model performance in the target domain by adjusting the distribution differences between the source domain (simulated image) and the target domain (real image) without target domain labels.

[0105] The training process of the initial model described above will be illustrated below with a specific example. Figure 4 As shown, the training process may include:

[0106] S1. The electronic device acquires the polarization sample image and RGB image of the sample glass collected by the line scan camera.

[0107] S2. The electronic device acquires a vector sample image of the sample glass.

[0108] S3. The electronic device inputs the polarization sample image, RGB image and vector sample image of the sample glass into the initial model to train the initial model.

[0109] The initial model training process may include:

[0110] S3a. The initial model performs frequency domain filtering on the polarization sample image and extracts polarization features from the filtered polarization sample image to obtain the polarization image features of the polarization sample image.

[0111] S3b: The initial model is based on the optical transmission model of the glass medium. The vector sample image is simulated to obtain the image features corresponding to the vector sample image.

[0112] S3c: The initial model performs spatial mapping between the image features corresponding to the vector sample image and the polarization image features of the polarization sample image to obtain a nonlinear spatial mapping matrix.

[0113] S3d and the initial model fuse the polarization image features of the polarization sample image and the RGB image to obtain the fused features.

[0114] This application constructs a dual-branch polarization feature extraction network, with branch 1 representing the original RGB image and branch 2 representing polarization image features. Based on this, feature fusion using a polarization-guided attention mechanism can be achieved.

[0115] S3e: The initial model performs multi-scale difference calculations based on the fused features and nonlinear spatial mapping matrix to obtain the defect probability map.

[0116] Multi-scale difference computation is also known as multi-scale difference feature computation. Optionally, illumination invariance constraints can be applied when performing multi-scale difference computation.

[0117] S3f: The initial model calculates the curvature continuity loss function based on the defect probability map, and then weights the curve continuity loss function and the preset loss function to obtain the multi-constraint joint loss function. The multi-constraint joint loss function is used to determine whether to continue training the initial model.

[0118] In some embodiments, the initial model described above can be implemented based on the MobileViT architecture and the high-resolution network (HRNet) architecture. The HRNet architecture maintains parallel feature propagation paths at four resolutions (1 / 1, 1 / 2, 1 / 4, 1 / 8).

[0119] The training process of the initial model has been described above. After obtaining the target model through training, it can be used to detect glass defects. The following section will further detail the results of glass defect detection using the target model.

[0120] In some embodiments, the electronic device can first simulate a vector image of the glass to be inspected to obtain simulated image features of the glass. Then, the electronic device can extract polarization features from the optical image of the glass to be inspected to obtain polarization image features of the glass. Finally, the electronic device inputs the simulated image features and the polarization image features of the glass to be inspected into a target model to obtain the defect detection result.

[0121] The process of determining the polarization image features and simulation image features mentioned above can be referred to the relevant description above, and will not be repeated here.

[0122] Furthermore, similar to the previous point, the simulated image features and polarization image features can be determined by or not by the target model. Accordingly, if the target model is determined, the electronic device inputs the optical image and vector image of the glass to be tested into the target model. Then, the electronic device simulates the vector image of the glass to be tested using the target model to obtain the simulated image features of the glass to be tested, and extracts polarization features from the optical image of the glass to be tested to obtain the polarization image features of the glass to be tested. In other words, the target model simulates the vector image of the glass to be tested and extracts polarization features from the optical image of the glass to be tested.

[0123] In this application, the target model compares the polarization image features of the glass to be tested with the simulated image features of the glass to be tested, that is, compares the actual style of the glass to be tested with the standard style, to determine whether the actual style of the glass to be tested is consistent with the standard style, thereby determining whether the glass to be tested has defects.

[0124] Optionally, the process of simulating the vector image of the glass to be tested includes:

[0125] Based on the glass medium light transmission model, the vector image of the glass to be tested is simulated; the glass medium light transmission model is determined based on the point spread function and the transmittance matrix to simulate the actual shooting situation of the standard pattern.

[0126] In some embodiments, similar to the preceding description, the process by which the target model determines the defect detection results may include:

[0127] Using the target model, the polarization image features of the glass to be tested and the color image of the glass to be tested are fused to obtain the fused features;

[0128] The target model determines the defect detection result based on the fused features and simulated image features. For example, the target model can perform multi-scale differential feature calculations based on the fused features and simulated image features to obtain a defect probability map. This defect probability map then determines whether the glass to be inspected has a defect, yielding the corresponding defect detection result. For instance, if the number of points in the defect probability map exceeding a preset probability threshold is greater than or equal to a preset number, it is determined that the glass to be inspected has a defect.

[0129] Alternatively, the color image of the glass to be tested used by the target model can also be input from an electronic device.

[0130] Optionally, when the target model performs multi-scale difference calculations (i.e., multi-scale difference feature calculations), it can use illumination invariance constraints as constraints.

[0131] It should be noted that the specific steps for determining the defect detection results using the target model are similar to those included in the training process described above. If steps such as simulation, polarization feature extraction, or feature fusion are required, this application does not impose any restrictions on these specific steps.

[0132] In some embodiments, since the target model identifies glass defects using vector and polarization images of the glass, it exhibits generalization ability. When detecting new types of glass, there is no need to retrain the target model; instead, the target model can be used directly for detection. Furthermore, this application incorporates the physical properties of polarized light into the attention mechanism to establish a physical constraint model for transparent medium defect detection to obtain the target model. This prevents misjudgments of defects caused by specific issues such as glass reflection, reduces the false detection rate, and thus ensures the accuracy of defect detection.

[0133] In this application, when it is necessary to detect defects in the glass to be tested, an optical image and a vector image of the glass to be tested are acquired. The vector image indicates a standard image of the glass to be tested, i.e., a standard pattern of the glass to be tested. Then, the optical image and vector image of the glass to be tested are used as parameters of a target model. The target model is run to perform comprehensive processing on the optical image and vector image, enabling the identification of defects in the glass to be tested using different types of images, obtaining the corresponding defect detection results, and realizing the detection of defects in transparent materials under optical physical constraints, ensuring the accuracy of defect detection. Furthermore, it can achieve accurate detection of minute defects, such as a detection rate of 99.3% for a 0.05mm ink spot defect. In addition, it improves the efficiency of glass defect detection.

[0134] It is understood that the operations performed by the models in this application, such as the initial model and the target model, are actually performed by the device on which the model resides, such as an electronic device.

[0135] In addition, the training process of the initial model can be performed by electronic devices or other devices. After training, the trained target model can be deployed on electronic devices to detect defects in the glass.

[0136] In some embodiments, the numbering of the above steps does not represent the actual execution order of the steps. The execution order of the above steps can be set according to requirements, and this application does not impose any restrictions on it.

[0137] To facilitate better implementation of the glass testing method provided in this application, this application also provides a glass testing device based on the above-described glass testing method. The meanings of the terms used are the same as in the glass testing method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0138] Figure 5 This is a schematic diagram of a glass inspection device provided in an embodiment of this application. Please refer to... Figure 5 This glass inspection device has the functionality to implement the method example described above on the electronic device side. This functionality can be implemented in hardware or by hardware executing corresponding software. The glass inspection device can be the electronic device described above, or it can be installed within an electronic device. For example... Figure 5 As shown, the glass inspection device 500 may include: an image acquisition module 510 and a defect detection module 520;

[0139] The image acquisition module 510 is used to acquire optical images and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates the standard image of the glass to be tested.

[0140] The defect detection module 520 is used to input the optical image and vector image of the glass to be tested into the target model to obtain the defect detection result corresponding to the glass to be tested.

[0141] In one example, the defect detection module 520 is specifically used for:

[0142] The vector image of the glass to be tested is simulated to obtain the simulated image features of the glass to be tested;

[0143] Polarization features are extracted from the optical image of the glass to be tested to obtain the polarization image features of the glass to be tested;

[0144] The simulated image features and polarization image features of the glass to be tested are input into the target model to obtain the defect detection results.

[0145] In one example, the defect detection module 520 is also specifically used for:

[0146] The vector image of the glass to be tested is simulated based on the glass medium optical transmission model; wherein the glass medium optical transmission model is determined based on the point spread function and the transmittance matrix.

[0147] In one example, the glass detection device described above also includes a training module.

[0148] The training module is used to acquire optical sample images and their corresponding vector sample images; and to train the initial model based on the optical sample images and their corresponding vector sample images to obtain the target model.

[0149] In one example, the training module is specifically used for:

[0150] The initial model is trained based on the optical sample images and their corresponding vector sample images;

[0151] Calculate the multi-constraint joint loss function corresponding to the trained model; wherein, the multi-constraint joint loss function is obtained by weighting the curvature continuity loss function and the preset loss function;

[0152] If the multi-constraint joint loss function is greater than the preset function value, return to the step of training the initial model based on the optical sample image and its corresponding vector sample image;

[0153] If the multi-constraint joint loss function is less than or equal to the preset function value, the trained model is used as the target model.

[0154] In one example, the curvature continuity loss function is determined based on the partial derivatives of pixels in the defect region corresponding to the optical sample image.

[0155] In one example, the defect region is determined based on a defect probability map;

[0156] The defect probability map is determined by multi-scale difference calculation of the image features corresponding to the optical sample image and the image features corresponding to the vector sample image.

[0157] In one example, the image features corresponding to the optical sample image are obtained by fusing the polarization image features of the optical sample image and the color image corresponding to the optical sample image;

[0158] The polarization image features of the optical sample image are obtained by extracting polarization features from the optical sample image.

[0159] The image features corresponding to the vector sample image are obtained by simulating the vector sample image.

[0160] Figure 6 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this application. Figure 6 The dashed line in the text indicates that the unit or module is optional. Figure 6 The electronic device 600 can be used to implement the methods described in the above method embodiments. The electronic device 600 can be a chip, a terminal device, or a first server.

[0161] Electronic device 600 may include one or more processors 610. The processor 610 can support the electronic device 600 in implementing the methods described in the preceding method embodiments. The processor 610 may be a general-purpose processor or a special-purpose processor. For example, the processor may be a Central Processing Unit (CPU). Alternatively, the processor may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0162] The electronic device 600 may also include one or more memories 620. Computer programs are stored on the memories 620. The memories 620 may be independent of the processor 610 or integrated into the processor 610.

[0163] Electronic device 600 may also include transceiver 630. Processor 610 can communicate with other devices or chips via transceiver 630. For example, processor 610 can send and receive data with other devices or chips via transceiver 630.

[0164] The computer program in memory 620 can be executed by processor 610, causing processor 610 to perform the following steps:

[0165] Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested;

[0166] The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

[0167] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0168] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, the computer program being loaded by a processor to execute the steps described in the above-described method embodiments of this application. For example, the computer program, when loaded by a processor, can execute the following steps:

[0169] Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested;

[0170] The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

[0171] For details on the implementation of each of the above operations / steps, please refer to the previous examples, which will not be repeated here.

[0172] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0173] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the above method embodiments provided in the embodiments of this application, the beneficial effects that the methods described in any of the above method embodiments can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0174] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0175] The glass detection method and electronic device provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A glass testing method, characterized in that, include: Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested; The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

2. The method according to claim 1, characterized in that, The step of inputting the optical image and vector image of the glass to be tested into the target model to obtain the defect detection result corresponding to the glass to be tested includes: The vector image of the glass to be tested is simulated to obtain the simulated image features of the glass to be tested; Polarization features are extracted from the optical image of the glass to be tested to obtain the polarization image features of the glass to be tested; The simulated image features and polarization image features of the glass to be tested are input into the target model to obtain the defect detection results.

3. The method according to claim 2, characterized in that, The simulation of the vector image of the glass to be tested includes: The vector image of the glass to be tested is simulated based on the glass medium optical transmission model; wherein the glass medium optical transmission model is determined based on the point spread function and the transmittance matrix.

4. The method according to any one of claims 1 to 3, characterized in that, Before inputting the optical image and vector image of the glass to be tested into the target model, the method further includes: Acquire optical sample images and their corresponding vector sample images; The initial model is trained based on the optical sample image and its corresponding vector sample image to obtain the target model.

5. The method according to claim 4, characterized in that, The step of training the initial model based on the optical sample image and its corresponding vector sample image to obtain the target model includes: The initial model is trained based on the optical sample images and their corresponding vector sample images; Calculate the multi-constraint joint loss function corresponding to the trained model; wherein, the multi-constraint joint loss function is obtained by weighting the curvature continuity loss function and the preset loss function; If the multi-constraint joint loss function is greater than the preset function value, return to the step of training the initial model based on the optical sample image and its corresponding vector sample image; If the multi-constraint joint loss function is less than or equal to the preset function value, the trained model is used as the target model.

6. The method according to claim 5, characterized in that, The curvature continuity loss function is determined based on the partial derivatives of the pixels in the defect region corresponding to the optical sample image.

7. The method according to claim 6, characterized in that, The defect area is determined based on a defect probability map; The defect probability map is determined by multi-scale difference calculation of the image features corresponding to the optical sample image and the image features corresponding to the vector sample image.

8. The method according to claim 7, characterized in that, The image features corresponding to the optical sample image are obtained by fusing the polarization image features of the optical sample image and the color image corresponding to the optical sample image. The polarization image features of the optical sample image are obtained by extracting polarization features from the optical sample image. The image features corresponding to the vector sample image are obtained by simulating the vector sample image.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores computer programs or instructions, and when the computer programs or instructions are executed by the processor, the processor causes the processor to perform the following steps: Acquire optical and vector images of the glass to be tested; wherein the vector image of the glass to be tested indicates a standard image of the glass to be tested; The optical image and vector image of the glass to be tested are input into the target model to obtain the defect detection results corresponding to the glass to be tested.

10. A computer-readable storage medium, characterized in that, It stores a computer program or instructions, which, when executed by a processor, implement the steps in the glass inspection method as described in any one of claims 1 to 8.

Images