A glass product defect automatic identification system and method

By combining optical and infrared detection technology, combined with X-ray transmission image analysis, and using deep neural network model, the problem of the inability to comprehensively detect glass product defects in the existing technology is solved, and accurate identification and automated judgment of surface and internal defects are achieved. Ensure product quality and safety.

CN118483254BActive Publication Date: 2024-11-12瞻阅(上海)新材料技术有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410948649.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-12
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

Existing glass product defect detection technology cannot fully detect surface and internal defects, especially internal structural defects, such as bubbles and cracks, and the defect type identification is inaccurate, making it difficult to achieve fully automated inspection.

Method used

Optical detection and infrared detection are combined to analyze surface defects through central light intensity and temperature distribution images, and internal defects are detected using X-ray transmission images, and defect type is automatically judged in combination with deep neural network models.

Benefits of technology

It realizes comprehensive inspection of the surface and internal defects of glass products, improves the accuracy and automation of defect identification, and ensures product quality and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118483254B_ABST
    Figure CN118483254B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of defect identification, and discloses a system and method for automatically identifying defects in glass products; the method comprises: collecting surface data of glass products; analyzing the surface data to determine whether there are defects on the surface of the glass products; identifying the defect type and defect position of glass products with defects on the surface; collecting internal data of the glass products; analyzing the internal data to determine whether there are defects inside the glass products; identifying the defect type and defect position of glass products with defects inside; the present invention greatly improves the accuracy and automation of defect identification; replaces the traditional inefficient detection method relying on manual labor, and realizes unmanned automatic detection of glass products throughout the process; can fully guarantee the product quality of glass products, and pays attention to product safety.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of defect recognition, and more specifically, to a system and method for automatically recognizing defects in glass products. Background Art

[0002] Glass products are indispensable materials in modern life and are widely used in construction, household goods, electronic products and other fields. However, various defects may occur during the production and processing, such as bubbles, cracks, scratches, etc. These defects not only affect the appearance and quality of the product, but may also threaten the safety of users. Traditional defect detection relies on manual visual inspection, which is not only inefficient but also prone to deviations. Therefore, how to quickly and accurately identify and detect defects in glass products has become an important research direction in glass product production and inspection.

[0003] The Chinese patent application with the authorization announcement number CN117783155B discloses a glass instrument defect detection system and method based on machine vision; it includes a data acquisition module: collecting light reflection data, light transmission data and light transmission image data of glass products through light sources, lighting equipment and image acquisition equipment; a defect positioning module: performing a comprehensive analysis of abnormalities in the entire area of ​​the glass products through light reflection and light transmission data to determine the location of the defect area; a defect profiling module: pre-processing the image data of the glass products, mapping the defect area and constructing a coordinate system to determine the coordinate position matrix of the defect area; constructing a defect recognition large model to judge the defect type of the glass products; a product feedback module: reporting and feedback on the real-time detection status of the glass products, and providing sound and light prompts for defective glass products; it can accurately locate and judge multiple types of glass defects while liberating manual productivity, thus realizing the automation and intelligence of glass defect detection;

[0004] However, the above technologies mainly rely on optical detection, including light reflection and light perspective detection, which can only detect surface defects and some superficial internal defects of glass products. It is unable to effectively monitor internal structural defects such as bubbles and cracks, making it difficult to fully control the quality of glass products, and thus unable to ensure the safety of glass products. At the same time, the large defect model used in the above technologies cannot accurately identify the defect type, and manual intervention is required for judgment, which reduces the detection efficiency and makes it difficult to achieve full automation of glass product detection.

[0005] In view of this, the present invention proposes a glass product defect automatic identification system and method to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for automatically identifying defects of glass products, comprising:

[0007] Collecting surface data of glass products; the surface data includes central light intensity and temperature distribution images;

[0008] Analyze the surface data to determine whether there are defects on the surface of the glass product; analyze the central light intensity to determine whether there are defects on the surface of the glass product, including: presetting a light difference threshold, calculating the difference between every two central light intensities, obtaining the light intensity difference, and comparing each light intensity difference with the light difference threshold to determine whether there are defects on the surface of the glass product; analyze the temperature distribution image to determine whether there are defects on the surface of the glass product, including: presetting a temperature difference threshold, calculating the temperature difference and the average temperature difference according to the temperature distribution image, and comparing each temperature difference and the average temperature difference with the temperature difference threshold to determine whether there are defects on the surface of the glass product;

[0009] Identify the defect type and location of glass products with surface defects;

[0010] Collect internal data of glass products;

[0011] Analyze internal data to determine whether there are defects inside the glass products;

[0012] Identify the defect type and location of glass products with internal defects.

[0013] Furthermore, the method for obtaining the central light intensity includes:

[0014] The surface of the glass product is divided into n areas, where n is an integer greater than 1; the same light is emitted to the center points of the n areas through a light source, and then the luminous intensity of each center point is collected by an optical sensor and marked as the central light intensity; the center point is the most central position point in the area.

[0015] Further, the difference between the light intensities of each two centers is calculated in turn to obtain The light intensity difference, ;

[0016] If each light intensity difference value is less than the light difference threshold, there is no defect on the surface of the glass product;

[0017] If there is a light intensity difference greater than or equal to the light difference threshold, there is a defect on the surface of the glass product;

[0018] According to the temperature distribution image, the temperature corresponding to each position point in each area is obtained; the temperature difference between every two position points in each area is calculated respectively; and the temperature difference of each area is compared with the temperature difference threshold value respectively;

[0019] If there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product;

[0020] If each temperature difference is less than the temperature difference threshold, count the number of location points in each area, add the temperatures of each location point in the same area in turn, and then divide it by the number of location points in the corresponding area to obtain the corresponding temperature mean of each area; calculate the difference between each two temperature means in turn to obtain The average temperature difference is calculated by comparing each average temperature difference with the temperature difference threshold; if each average temperature difference is less than the temperature difference threshold, there is no defect on the surface of the glass product; if there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product.

[0021] Furthermore, the method for identifying defect positions of glass products with surface defects includes:

[0022] Mark the area where the light intensity difference is greater than or equal to the light difference threshold as a defect area; obtain the luminous intensity of each position point in each defect area; calculate the luminous intensity difference between every two position points in the same defect area in turn, and compare each luminous intensity difference with the light difference threshold in turn; if the luminous intensity difference is greater than or equal to the light difference threshold, compare the luminous intensity of the corresponding two position points, mark the position point with large luminous intensity as the first defect point, and do not mark the position point with small luminous intensity; if the luminous intensity difference is less than the light difference threshold, do not mark the corresponding two position points;

[0023] Grayscale the temperature distribution image, collect the grayscale values ​​of Y position points, where Y is the number of all position points in the temperature distribution image; compare the grayscale values ​​of the Y position points with the grayscale value threshold respectively, mark the position points with grayscale values ​​greater than or equal to the grayscale value threshold as the second defect point, and do not mark the position points with grayscale values ​​less than the grayscale value threshold;

[0024] In the temperature distribution image, the position point marked as the first defect point according to the luminous intensity is also marked as the first defect point, and one position point can be marked as the first defect point and the second defect point at the same time;

[0025] A plurality of adjacent first defect points are regarded as a first surface defect, and a plurality of adjacent second defect points are regarded as a second surface defect; a first defect point that is adjacent to both the position point and the first defect point is marked as a first contour point, and a first defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all first defect points is also marked as a first contour point; similarly, a second defect point that is adjacent to both the position point and the second defect point is marked as a second contour point, and a second defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all second defect points is also marked as a second contour point;

[0026] A rectangular coordinate system is established with any vertex of the temperature distribution image as the origin, and the coordinates corresponding to each first contour point are obtained and marked as first contour coordinates, and the coordinates corresponding to each second contour point are obtained and marked as second contour coordinates; if the coordinates of the first contour point corresponding to the first surface defect are the same as the coordinates of the second contour point corresponding to the second surface defect, the corresponding first surface defect and the second surface defect are marked as surface defects, and the corresponding first contour coordinates and the second contour coordinates are marked as contour coordinates; a scaling factor is preset, and the first contour coordinates are multiplied by the scaling factor after square root to obtain the first actual coordinates, and the second contour coordinates are multiplied by the scaling factor after square root to obtain the second actual coordinates, and the contour coordinates are multiplied by the scaling factor after square root to obtain the actual coordinates; the position of each first surface defect is identified according to the first actual coordinates, the position of each second surface defect is identified according to the second actual coordinates, and the position of each surface defect is identified according to the actual coordinates.

[0027] Furthermore, the method for identifying the defect type of a glass product having defects on the surface includes:

[0028] The first surface defect, the second surface defect and the surface defect are regarded as product surface defects; the number of position points in each product surface defect is counted, and the defect area of ​​each product surface defect is calculated; the expression of the defect area is: ; In the formula, is the defect area, is the number of position points, is the area of ​​a location point, is the scaling factor;

[0029] The area of ​​a location point is obtained by the resolution of the temperature distribution image, and the resolution of the temperature distribution image is obtained by the attributes of the temperature distribution image;

[0030] The first actual coordinate, the second actual coordinate and the actual coordinate are used as defect contour coordinates; different digital labels are set for different product surface defects and marked as defect labels; a defect label, a defect area corresponding to the defect label and a group of defect contour coordinates corresponding to the defect label are used as a group of analysis data; each group of analysis data is input into the trained type prediction model to identify the corresponding defect type; the training process of the type prediction model includes:

[0031] Pre-setting corresponding recognition results for a group of analysis data, where a is an integer greater than 1, and setting different digital labels for different recognition results; marking the digital labels of the recognition results as recognition labels, and converting the analysis data and the corresponding recognition labels into a corresponding set of feature vectors;

[0032] Each group of feature vectors is used as the input of the type prediction model. The type prediction model takes a group of predicted identification labels corresponding to each group of analysis data as output, and takes the actual identification labels corresponding to each group of analysis data as prediction targets, where the actual identification labels are the digital labels of the identification results corresponding to the analysis data collected in advance. Minimizing the sum of prediction errors of all analysis data is used as the training target. The type prediction model is trained until the sum of prediction errors reaches convergence and the training is stopped. The type prediction model is a deep neural network model. According to the obtained identification labels, the corresponding defect types are obtained.

[0033] Further, the internal data is an X-ray transmission image;

[0034] Methods for obtaining internal data include:

[0035] X-rays are emitted through an X-ray source, and then the X-ray intensity distribution passing through the glass product is recorded through a planar X-ray detector to obtain an X-ray intensity distribution image; the relative positions of the X-ray source and the planar X-ray detector are rotated in sequence, that is, each rotation is b°, b is a real number greater than 0, to obtain different irradiation angles; multiple X-ray intensity distribution images are collected from different irradiation angles, and the X-ray intensity of each pixel point in each X-ray intensity distribution image is digitally saved as a three-dimensional digital matrix with pixel point coordinates, that is, each X-ray intensity distribution image is converted into a three-dimensional digital matrix; and an X-ray transmission image is constructed according to each three-dimensional digital matrix using a reconstruction algorithm.

[0036] Furthermore, the analyzing of the internal data to determine whether there are defects inside the glass product includes: using a trained defect determination model to determine the X-ray transmission image and outputting a determination result, wherein the determination result includes whether there are defects or not;

[0037] The defect judgment model training process includes:

[0038] Collect c X-ray transmission images in advance, annotate each X-ray transmission image, and the annotations include the presence of defects and the absence of defects; convert the presence of defects and the absence of defects into digital annotations respectively; divide the annotated X-ray transmission images into a training set and a test set; use the training set to train the defect judgment model, and use the test set to test the defect judgment model; preset an error threshold, and when the mean of the prediction errors of all X-ray transmission images in the test set is less than the error threshold, output the defect judgment model; the defect judgment model is a convolutional neural network model.

[0039] Furthermore, the method for identifying the defect position of a glass product having internal defects includes:

[0040] Using the trained defect annotation model, the defect area in the X-ray transmission image is annotated with a three-dimensional rectangular frame; the eight vertex coordinates of each three-dimensional rectangular frame are obtained, and the position of each internal defect is identified according to the eight vertex coordinates of each three-dimensional rectangular frame;

[0041] The training process of the defect annotation model includes:

[0042] Pre-collect d X-ray transmission images, where d is an integer greater than 1, pre-identify the defect area in each X-ray transmission image, and annotate the defect area in the X-ray transmission image with a three-dimensional rectangular frame; divide the annotated X-ray transmission images into a training set and a test set, use 70% of the X-ray transmission images as the training set, and use 30% of the X-ray transmission images as the test set, use the training set to train the defect annotation model, and use the test set to test the defect annotation model; preset a coordinate error threshold, and when the mean of the predicted coordinate errors of all X-ray transmission images in the test set is less than the coordinate error threshold, the defect annotation model training is completed, and the defect annotation model is output; the defect annotation model is a convolutional neural network model.

[0043] Furthermore, the method for identifying the defect type of a glass product having internal defects includes:

[0044] According to the marked three-dimensional rectangular frame, the X-ray transmission image is cropped to obtain f defect images, where f is the number of three-dimensional rectangular frames; the X-ray intensities of H pixels in each defect image are collected, where H is the number of all pixels in each defect image; an intensity range value is preset, and the X-ray intensities of the H pixels in each defect image are compared with the intensity range value respectively, and the pixel point whose X-ray intensity is greater than the maximum value of the intensity range value is marked as the third defect point, and the pixel point whose X-ray intensity is less than the minimum value of the intensity range value is marked as the fourth defect point, and the pixel point whose X-ray intensity is within the intensity range is not marked;

[0045] The defect image with the third defect point is marked as a high-density image, and the image with the fourth defect point is marked as a low-density image; the grid volume integration algorithm is used to calculate the defect volume of each high-density image and low-density image; different numbers are set for high-density images and low-density images, and marked as image numbers; an image number, a defect volume and a set of vertex coordinates of a three-dimensional rectangular box are used as a set of test data, and each set of test data is input into the trained type analysis model to obtain the corresponding defect type; the training process of the type analysis model is consistent with the training process of the type prediction model, and it is also a deep neural network model.

[0046] Furthermore, the method further comprises: collecting the ambient temperature, and compensating the temperature in the temperature distribution image according to the ambient temperature;

[0047] The method for compensating the temperature in the temperature distribution image according to the ambient temperature includes:

[0048] Preset the normal temperature, subtract the normal temperature from the ambient temperature, and obtain the temperature deviation;

[0049] According to the temperature deviation and the temperature distribution image, the deviation probability corresponding to each temperature in the temperature distribution image is calculated; the expression of the deviation probability is: ; In the formula is the deviation probability corresponding to the i-th temperature in the temperature distribution image, is the temperature deviation, is the i-th temperature in the temperature distribution image, , e is a natural constant;

[0050] If the deviation probability corresponding to the temperature is greater than or equal to , then the temperature is marked as the temperature to be compensated; if the deviation probability corresponding to the temperature is less than , the temperature is not marked as the temperature to be compensated; is a real number greater than 1;

[0051] The temperature compensation amount is calculated based on the temperature to be compensated and the temperature deviation; the expression of the temperature compensation amount is: ; In the formula, is the temperature compensation corresponding to the jth temperature in the temperature distribution image, is the jth temperature in the temperature distribution image, , is the number of temperatures to be compensated, , is the preset weight coefficient;

[0052] The corresponding temperature compensation amount is subtracted from each temperature to be compensated to obtain the compensated temperature, and the temperatures to be compensated in the temperature distribution image are replaced with the corresponding compensated temperatures to complete the temperature compensation.

[0053] A glass product defect automatic identification system, used to implement the glass product defect automatic identification method, comprising:

[0054] A surface data acquisition module is used to collect surface data of glass products;

[0055] The surface defect judgment module is used to analyze the surface data and judge whether there are defects on the surface of the glass product;

[0056] A surface defect recognition module is used to identify the defect type and defect location of glass products with surface defects;

[0057] Internal data collection module, used to collect internal data of glass products;

[0058] Internal defect judgment module, used to analyze internal data and judge whether there are defects inside the glass product;

[0059] The internal defect recognition module is used to identify the defect type and defect location of glass products with internal defects.

[0060] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the automatic identification method for defects of glass products when executing the computer program.

[0061] A computer-readable storage medium stores a computer program, and when the computer program is executed, the method for automatically identifying defects in glass products is implemented.

[0062] The technical effects and advantages of the automatic identification system and method of glass product defects of the present invention are as follows:

[0063] 1. By simultaneously performing optical and infrared inspections on the surface of glass products, as well as X-ray inspections on the inside of glass products, it is possible to detect both superficial and deep defects on the surface of glass products, as well as structural defects inside glass products, fundamentally improving the integrity of defect detection in glass products. It uses a variety of recognition methods to classify and quantitatively identify defects, accurately identify defect locations, and use deep learning models to automatically determine defect types, greatly improving the accuracy and automation of defect identification. It replaces the traditional inefficient detection method that relies on manual labor, and realizes unmanned automatic inspection of glass products throughout the process. It can fully guarantee the product quality of glass products, focus on product safety, and provide technical support for the widespread application of glass products.

[0064] 2. Utilize the deviation between ambient temperature and normal temperature to calculate the deviation probability of the temperature at each location point, calculate the corresponding temperature compensation amount for the temperature that needs to be compensated, and effectively perform temperature compensation on the temperature distribution image in a high temperature environment, reducing the problem of reduced temperature difference caused by increased ambient temperature; it can not only dynamically adapt to changes in ambient temperature and improve the reliability and stability of the recognition system, but also ensure the consistency of defect recognition results in different environments, thereby significantly improving the accuracy and efficiency of automatic recognition of defects in glass products. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a schematic diagram of an automatic glass product defect recognition system according to Example 1 of the present invention;

[0066] Figure 2 This is a schematic diagram of an automatic glass product defect recognition system according to Embodiment 2 of the present invention;

[0067] Figure 3 This is a schematic diagram of a process flow of a method for automatically identifying defects in glass products according to Embodiment 3 of the present invention;

[0068] Figure 4 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention;

[0069] Figure 5 This is a schematic diagram of a storage medium according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Example 1

[0072] See also Figure 1 As shown, the automatic identification system for glass product defects described in this embodiment includes a surface data acquisition module, a surface defect judgment module, a surface defect recognition module, an internal data acquisition module, an internal defect judgment module and an internal defect recognition module; each module is connected by wire and / or wireless means to realize data transmission between modules;

[0073] A surface data acquisition module, used to collect surface data of glass products;

[0074] Surface data include central light intensity and temperature distribution images;

[0075] Methods for obtaining the central light intensity include:

[0076] The surface of the glass product is divided into n areas, where n is an integer greater than 1; the light source installed above the defect recognition platform emits the same light to the center points of the n areas, and then the optical sensor installed above the defect recognition platform collects the luminous intensity of each center point and marks it as the center light intensity; wherein the defect recognition platform is a platform for identifying defects in glass products, the center point is the most central position point in the area, and the light source and the optical sensor are both movable, and the luminous intensity of the surface of the glass product is fully collected by moving;

[0077] The temperature distribution image is acquired by an infrared array sensor installed above the defect recognition platform.

[0078] It should be noted that when the surface of a glass product is smooth and flat without any defects, no significant optical changes such as reflection and refraction will occur when light propagates on the surface of the glass product. Therefore, the luminous intensity at different positions on the surface of the glass product is basically the same, and the variation range is very small. If the luminous intensity at a few positions on the surface of the glass product is significantly different from that at most other positions, it means that there are defects on the surface of the glass product. Similarly, when the surface of a glass product is smooth and flat without any defects, different positions on the surface of the glass product will not accumulate or lose heat unevenly due to differences in physical structure. Therefore, the temperature at different positions on the surface of the glass product should also be balanced, that is, the temperature difference is very small. If there is a significant temperature difference between a few positions and most positions on the surface of the glass product, it means that the physical structure of the surface of the glass product has changed, that is, there are defects on the surface of the glass product.

[0079] The surface defect judgment module is used to analyze the surface data and judge whether there are defects on the surface of the glass product;

[0080] Methods for analyzing the central light intensity to determine whether there are defects on the surface of glass products include:

[0081] Calculate the difference between the light intensities of each two centers in turn to obtain The light intensity difference, ; Preset the light difference threshold, and compare each light intensity difference value with the light difference threshold;

[0082] If each light intensity difference is less than the light difference threshold, there is no defect on the surface of the glass product, which means that the difference in luminous intensity at different positions on the surface of the glass product is small, and no significant optical changes such as reflection and refraction occur;

[0083] If there is a light intensity difference greater than or equal to the light difference threshold, there is a defect on the surface of the glass product, which means that the luminous intensity of at least one center point on the surface of the glass product is significantly different from the luminous intensity of the other center points, that is, significant optical changes such as reflection and refraction have occurred on the surface of the glass product;

[0084] It should be noted that the light difference threshold is determined by technical personnel in this field during the historical process of identifying defects in glass products. They obtain multiple defective glass products, collect the central light intensity of each glass product, and calculate the central light intensity difference between the central light intensity at the defect location and the central light intensity at each normal location in each glass product. The minimum value of the central light intensity difference corresponding to each glass product is obtained and marked as the minimum difference, and the average of multiple minimum differences is used as the light threshold.

[0085] Methods for analyzing the temperature distribution image to determine whether there are defects on the surface of the glass product include:

[0086] According to the temperature distribution image, the temperature corresponding to each position point in each area is obtained; the temperature difference between every two position points in each area is calculated respectively, and the position points are the pixel points in the temperature distribution image; the temperature difference threshold is preset, and the temperature difference of each area is compared with the temperature difference threshold;

[0087] If the temperature difference is greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product, indicating that there is a large temperature difference between at least one point on the surface of the glass product and the other points in the same area, that is, there is a phenomenon of uneven accumulation or loss of heat;

[0088] If each temperature difference is less than the temperature difference threshold, count the number of location points in each area, add the temperatures of each location point in the same area in turn, and then divide it by the number of location points in the corresponding area to obtain the corresponding temperature mean of each area; calculate the difference between each two temperature means in turn to obtain average temperature difference; then compare each average temperature difference with the temperature difference threshold; if each average temperature difference is less than the temperature difference threshold, there is no defect on the surface of the glass product, indicating that the temperature at different positions on the surface of the glass product is balanced; if there is an average temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product, indicating that there is a large difference between the temperature of all positions in at least one area on the surface of the glass product and the temperature of positions in other areas.

[0089] It should be noted that the preset method of the temperature difference threshold is consistent with the preset method of the light difference threshold; the reason for calculating the temperature mean and the temperature mean difference to determine whether the surface of the glass product has defects is that there may be a region on the surface of the glass product with defects. In this case, each temperature difference in the region is less than the temperature difference threshold. Therefore, if only the temperature difference in each region is used to determine whether the surface of the glass product has defects, it will cause misjudgment.

[0090] It should be understood that the reason for using both optical and infrared inspections to inspect the surface of glass products is that there may be small and shallow scratches on the surface of glass products, which have little effect on the temperature of the surface of the glass products. It is difficult to detect such defects using only infrared inspection, so optical inspection is required; and there may be internal defects such as bubbles and interlayers below the surface of the glass products. It is difficult to detect such defects using only optical inspection, so infrared inspection is required; therefore, it is impossible to fully detect defects on the surface of glass products using only one of optical inspection or infrared inspection, and both inspection methods need to be used simultaneously.

[0091] A surface defect recognition module is used to identify the defect type and defect location of glass products with surface defects;

[0092] Methods for identifying defect locations of glass products with surface defects include:

[0093] Mark the area where the light intensity difference is greater than or equal to the light difference threshold as a defect area; obtain the luminous intensity of each position point in each defect area, where the method for obtaining the luminous intensity of each position point is consistent with the method for obtaining the central light intensity; calculate the luminous intensity difference between every two position points in the same defect area in turn, and compare each luminous intensity difference with the light difference threshold in turn; if the luminous intensity difference is greater than or equal to the light difference threshold, compare the luminous intensities of the corresponding two position points, mark the position point with large luminous intensity as the first defect point, and do not mark the position point with small luminous intensity; if the luminous intensity difference is less than the light difference threshold, do not mark the corresponding two position points;

[0094] The temperature distribution image is grayed, and the gray values ​​of Y position points are collected, where Y is the number of all position points in the temperature distribution image; the gray values ​​of the Y position points are compared with the gray value threshold respectively, and the position points with gray values ​​greater than or equal to the gray value threshold are marked as second defect points, and the position points with gray values ​​less than the gray value threshold are not marked; the gray value threshold is obtained by a technician in the field of glass product defect identification in the past, and multiple temperature distribution images of glass products with surface defects are obtained and grayed, and the gray values ​​of the second defect points in each temperature distribution image are collected and marked as defect gray values, and the defect gray value with the smallest value in each temperature distribution image is obtained and marked as the minimum gray value, and the average of multiple minimum gray values ​​is used as the gray value threshold;

[0095] In the temperature distribution image, the position point marked as the first defect point according to the luminous intensity is also marked as the first defect point, and one position point can be marked as the first defect point and the second defect point at the same time; a plurality of adjacent first defect points are regarded as a first surface defect, and a plurality of adjacent second defect points are regarded as a second surface defect; a first defect point that is adjacent to both the position point and the first defect point is marked as a first contour point, and a first defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all first defect points is also marked as a first contour point; similarly, a second defect point that is adjacent to both the position point and the second defect point is marked as a second contour point, and a second defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all second defect points is also marked as a second contour point;

[0096] A rectangular coordinate system is established with any vertex of the temperature distribution image as the origin, and the coordinates corresponding to each first contour point are obtained and marked as the first contour coordinates, and the coordinates corresponding to each second contour point are obtained and marked as the second contour coordinates; wherein the vertices are the pixel points corresponding to the upper left corner, the upper right corner, the lower left corner and the lower right corner in the temperature distribution image; if the coordinates of the first contour point corresponding to the first surface defect are the same as the coordinates of the second contour point corresponding to the second surface defect, then the corresponding first surface defect and the second surface defect are marked as surface defects, and the corresponding first contour coordinates and the second contour coordinates are marked as contour coordinates, indicating that the surface defect is both a first surface defect and a second surface defect; a scaling factor is preset, and the first contour coordinates are multiplied by the scaling factor after the square root in turn to obtain the first actual coordinates, and the second contour coordinates are multiplied by the scaling factor after the square root in turn to obtain the second actual coordinates, and the contour coordinates are multiplied by the scaling factor after the square root in turn to obtain the actual coordinates; the position of each first surface defect is identified according to the first actual coordinates, the position of each second surface defect is identified according to the second actual coordinates, and the position of each surface defect is identified according to the actual coordinates;

[0097] When determining the grayscale value threshold, the scaling factor is determined by a technician in this field by measuring the surface area of ​​the glass product in multiple temperature distribution images and the actual surface area of ​​the glass product, dividing the actual surface area of ​​the glass product by the surface area of ​​the glass product in the temperature distribution image to obtain a quotient, and taking the average of the multiple quotients as the scaling factor.

[0098] Methods for identifying defect types of glass products with surface defects include:

[0099] The first surface defect, the second surface defect and the surface defect are regarded as product surface defects; the number of position points in each product surface defect is counted, and the defect area of ​​each product surface defect is calculated; the expression of the defect area is: ; In the formula, is the defect area, is the number of position points, is the area of ​​a location point, is the scaling factor;

[0100] The area of ​​a location point is obtained by the resolution of the temperature distribution image, and the resolution of the temperature distribution image is obtained by the attributes of the temperature distribution image;

[0101] The first actual coordinate, the second actual coordinate and the actual coordinate are used as defect contour coordinates; different digital labels are set for different product surface defects and marked as defect labels; a defect label, a defect area corresponding to the defect label and a group of defect contour coordinates corresponding to the defect label are used as a group of analysis data; each group of analysis data is input into the trained type prediction model to identify the corresponding defect type;

[0102] The specific training process of the type prediction model includes:

[0103] A corresponding recognition result is set in advance for a group of analysis data, where a is an integer greater than 1, and the recognition result is, for example, a bubble, a crack, a dent, etc. Different digital labels are set for different recognition results. For example, a digital label is set to 1 for a bubble, a digital label is set to 2 for a crack, and a digital label is set to 3 for a dent. The recognition result corresponding to the analysis data is collected by a person skilled in the art during the historical glass product defect recognition process. The person skilled in the art sequentially identifies the defect type existing on the surface of the glass product under the condition of each group of analysis data based on experience, and sequentially sets corresponding recognition results for multiple groups of different analysis data;

[0104] Marking the digital labels of the recognition results as recognition labels, and converting the analysis data and the corresponding recognition labels into a corresponding set of feature vectors;

[0105] Each set of feature vectors is used as the input of the type prediction model. The type prediction model takes a set of predicted identification labels corresponding to each set of analysis data as output, and takes the actual identification labels corresponding to each set of analysis data as the prediction target. The actual identification labels are the digital labels of the recognition results corresponding to the analysis data collected in advance. The training goal is to minimize the sum of the prediction errors of all analysis data. The calculation formula of the prediction error is: ,in is the prediction error, is the group number of the eigenvector corresponding to the analyzed data, For the The predicted identification labels corresponding to the group analysis data, For the The actual identification label corresponding to the group analysis data; the type prediction model is trained until the sum of the prediction errors reaches convergence and the training is stopped; the type prediction model is specifically a deep neural network model; according to the obtained identification label, the corresponding defect type is obtained.

[0106] Internal data collection module, used to collect internal data of glass products;

[0107] The internal data is X-ray transmission images;

[0108] Methods for obtaining internal data include:

[0109] X-rays are emitted by an X-ray source installed on the side of the defect identification platform, and then the X-ray intensity distribution passing through the glass product is recorded by a planar X-ray detector installed on the other side of the defect identification platform to obtain an X-ray intensity distribution image. The X-ray emission source and the planar X-ray detector are distributed on both sides of the glass product; the relative positions of the X-ray source and the planar X-ray detector are rotated in sequence, that is, each rotation is b°, b is a real number greater than 0, to obtain different irradiation angles to fully cover the entire glass product; multiple X-ray intensity distribution images are collected from different irradiation angles, and the X-ray intensity of each pixel point in each X-ray intensity distribution image is digitally saved as a three-dimensional digital matrix with pixel point coordinates, that is, each X-ray intensity distribution image is converted into a three-dimensional digital matrix; a reconstruction algorithm (such as FBP algorithm, SART algorithm) is used according to each three-dimensional digital matrix to construct an X-ray transmission image, and the X-ray transmission image is a three-dimensional image.

[0110] Internal defect judgment module, used to analyze internal data and judge whether there are defects inside the glass product;

[0111] Use the trained defect judgment model to judge the X-ray transmission image and output the judgment result, which includes whether there is a defect or not;

[0112] The specific training process of the defect judgment model includes:

[0113] Collect c X-ray transmission images in advance, annotate each X-ray transmission image, and the annotation includes whether there are defects or not; convert the presence of defects and the absence of defects into digital annotations, for example, the presence of defects is converted to 10, and the absence of defects is converted to 20; divide the annotated X-ray transmission images into a training set and a test set, use 70% of the X-ray transmission images as the training set, and use 30% of the X-ray transmission images as the test set; use the training set to train the defect judgment model, and use the test set to test the defect judgment model; preset an error threshold, and when the mean of the prediction errors of all X-ray transmission images in the test set is less than the error threshold, output the defect judgment model; wherein the calculation formula for the mean prediction error is: ,in is the prediction error, is the number of the X-ray transmission image, For the The predicted annotations corresponding to the set of X-ray transmission images, For the The actual annotations corresponding to the group of X-ray transmission images, U is the number of X-ray transmission images in the test set;

[0114] The error threshold is pre-set according to the accuracy required by the defect judgment model; the defect judgment model is specifically a convolutional neural network model.

[0115] Internal defect recognition module, used to identify the defect type and defect location of glass products with internal defects;

[0116] Methods for identifying defect locations of glass products with internal defects include:

[0117] Use the trained defect annotation model to annotate the defect area in the X-ray transmission image with a three-dimensional rectangular box; obtain the eight vertex coordinates of each three-dimensional rectangular box, and identify the position of each internal defect according to the eight vertex coordinates of each three-dimensional rectangular box; the eight vertex coordinates of the three-dimensional rectangular box can be obtained through an image processing library (such as OpenCV, PIL, scikit-image, etc.);

[0118] The training process of the defect annotation model includes:

[0119] Collect d X-ray transmission images in advance, where d is an integer greater than 1, and the defect area in each X-ray transmission image is identified in advance by a technician in this field, and the defect area in the X-ray transmission image is annotated with a three-dimensional rectangular frame; the annotated X-ray transmission images are divided into a training set and a test set, 70% of the X-ray transmission images are used as the training set, and 30% of the X-ray transmission images are used as the test set, and the defect annotation model is trained using the training set, and the defect annotation model is tested using the test set; a coordinate error threshold is preset, and when the mean value of the predicted coordinate errors of all X-ray transmission images in the test set is less than the coordinate error threshold, the defect annotation model training is completed, and the defect annotation model is output; wherein the calculation formula for the mean value of the predicted coordinate error includes:

[0120] ;

[0121] in, is the predicted coordinate error, is the number of X-ray transmission images, T is the number of X-ray transmission images in the test set, , is the predicted horizontal coordinate of a vertex, is the actual horizontal coordinate of a vertex, is the predicted ordinate of a vertex, is the actual ordinate of a vertex, is the number of the vertices of the three-dimensional rectangular box, and the three-dimensional rectangular box has 8 vertices in total; the coordinate error threshold is pre-set according to the accuracy required by the defect annotation model; the defect annotation model is specifically a convolutional neural network model.

[0122] Methods for identifying defect types of glass products with internal defects include:

[0123] According to the marked three-dimensional rectangular frame, the X-ray transmission image is cropped to obtain f defect images, where f is the number of three-dimensional rectangular frames; the X-ray intensities of H pixels in each defect image are collected, where H is the number of all pixels in each defect image; the intensity range value is preset, and the X-ray intensities of the H pixels in each defect image are compared with the intensity range value respectively, and the pixel point whose X-ray intensity is greater than the maximum value of the intensity range value is marked as the third defect point, and the pixel point whose X-ray intensity is less than the minimum value of the intensity range value is marked as the fourth defect point, and the pixel point whose X-ray intensity is within the intensity range is not marked. ; The intensity range value is obtained by a technician in the field of glass product defect identification in the past. The defect images of multiple glass products with internal defects are obtained, and the X-ray intensity of the third defect point in the defect image is collected and marked as the third intensity. The X-ray intensity of the fourth defect point in the defect image is collected and marked as the fourth intensity. The third intensity with the smallest value in each defect image is obtained and marked as the minimum intensity. The fourth intensity with the largest value in each defect image is obtained and marked as the maximum intensity. The average of multiple minimum intensities is used as the maximum value of the intensity range value, and the average of multiple maximum intensities is used as the minimum value of the intensity range value.

[0124] It should be noted that if the defects in the defect image are low-density defects such as voids and bubbles, the corresponding X-ray intensity will be lower than that of pixels without defects; if the defects in the defect image are high-density defects such as impurities and grains, the corresponding X-ray intensity will be higher than that of pixels without defects;

[0125] The defective image with the third defect point is marked as a high-density image, and the image with the fourth defect point is marked as a low-density image; the grid volume integration algorithm is used to calculate the defect volume of each high-density image and low-density image. The grid volume integration algorithm is a prior art and will not be described in detail here; different numbers are set for high-density images and low-density images, and marked as image numbers, for example, the high-density image is numbered 100 and the low-density image is numbered 200; an image number, a defect volume and a set of three-dimensional rectangular box vertex coordinates are used as a set of test data, and each set of test data is input into the trained type analysis model to obtain the corresponding defect type; the specific training process of the type analysis model is consistent with the training process of the type prediction model, and it is also a deep neural network model.

[0126] This embodiment simultaneously performs optical inspection and infrared inspection on the surface of glass products, and performs X-ray inspection on the inside of glass products, so as to detect both superficial and deep defects on the surface of glass products and structural defects inside glass products, thereby fundamentally improving the integrity of glass product defect detection; utilizes a variety of recognition methods to classify and quantitatively identify defects, accurately identifies defect locations, and uses deep learning models to automatically determine defect types, greatly improving the accuracy and automation of defect recognition; replaces traditional inefficient detection methods that rely on manual labor, and realizes unmanned automatic inspection of glass products throughout the process; can fully guarantee the product quality of glass products, focus on product safety, and provide technical support for the widespread application of glass products.

[0127] Example 2

[0128] See also Figure 2 As shown, this embodiment is further improved on the basis of the first embodiment. When judging whether there is a defect on the surface of the glass product according to the temperature distribution image, if the temperature of the environment is high, the temperature of each position point in the temperature distribution image will rise, and the degree of rise is different, thereby causing the temperature difference between the second defect point and the position point not marked as the second defect point to decrease. If the reduced temperature difference is less than the temperature difference threshold, it will cause a misjudgment. Therefore, this embodiment provides a glass product defect automatic identification system, which also includes a temperature compensation module;

[0129] A temperature compensation module is used to collect the ambient temperature and compensate the temperature in the temperature distribution image according to the ambient temperature;

[0130] The ambient temperature is the temperature of the glass product defect identification site, which is the site where the automatic identification of glass product defects is performed; the ambient temperature is obtained by a temperature sensor installed in the glass product defect identification site;

[0131] The method for compensating the temperature in the temperature distribution image according to the ambient temperature includes:

[0132] The normal temperature is preset. The normal temperature is obtained by a technician in the field of glass product defect recognition in the past, and the corresponding ambient temperatures when multiple glass product defects are correctly identified are obtained. The correct identification of glass product defects means that the defects identified by the automatic defect recognition system are consistent with the defects identified by the technician in the field of glass product defect recognition. The average of multiple ambient temperatures is taken as the normal temperature.

[0133] Subtract the normal temperature from the ambient temperature to obtain the temperature deviation;

[0134] According to the temperature deviation and the temperature distribution image, the deviation probability corresponding to each temperature in the temperature distribution image is calculated; the expression of the deviation probability is: ; In the formula is the deviation probability corresponding to the i-th temperature in the temperature distribution image, is the temperature deviation, is the i-th temperature in the temperature distribution image, , e is a natural constant;

[0135] If the deviation probability corresponding to the temperature is greater than or equal to , the temperature is marked as the temperature to be compensated, indicating that the probability of temperature deviation is high and compensation is required to avoid misjudgment; if the deviation probability corresponding to the temperature is less than , the temperature is not marked as the temperature to be compensated, which means that the probability of temperature deviation is small and no compensation is required; is a real number greater than 1, and this embodiment preferably is 4;

[0136] The temperature compensation amount is calculated based on the temperature to be compensated and the temperature deviation; the expression of the temperature compensation amount is: ; In the formula, is the temperature compensation corresponding to the jth temperature in the temperature distribution image, is the jth temperature in the temperature distribution image, , is the number of temperatures to be compensated, , is a preset weight coefficient; the specific value of the weight coefficient can be set according to the actual situation, and the weight coefficient reflects the influence of the temperature to be compensated and the temperature deviation on the temperature compensation amount. Those skilled in the art can preset the corresponding weight coefficient according to the actual influence of the temperature to be compensated and the temperature deviation on the temperature compensation amount, so as to accurately evaluate the temperature compensation amount;

[0137] The corresponding temperature compensation amount is subtracted from each temperature to be compensated to obtain the compensated temperature, and the temperatures to be compensated in the temperature distribution image are replaced with the corresponding compensated temperatures to complete the temperature compensation.

[0138] This embodiment uses the deviation between the ambient temperature and the normal temperature to calculate the deviation probability of the temperature at each location point, and calculates the corresponding temperature compensation amount for the temperature that needs to be compensated, thereby effectively performing temperature compensation on the temperature distribution image in a high-temperature environment, reducing the problem of reduced temperature difference caused by increased ambient temperature; it can not only dynamically adapt to changes in ambient temperature and improve the reliability and stability of the recognition system, but also ensure that the defect recognition results in different environments are consistent, thereby significantly improving the accuracy and efficiency of automatic recognition of defects in glass products.

[0139] Example 3

[0140] See also Figure 3As shown, the part not described in detail in this embodiment can be seen from the description of embodiment 1 and embodiment 2. A method for automatically identifying defects in glass products is provided, and the method includes:

[0141] Collect surface data of glass products;

[0142] Analyze the surface data to determine whether there are defects on the surface of the glass product;

[0143] Identify the defect type and location of glass products with surface defects;

[0144] Collect internal data of glass products;

[0145] Analyze internal data to determine whether there are defects inside the glass products;

[0146] Identify the defect type and location of glass products with internal defects.

[0147] Further, the surface data includes central light intensity and temperature distribution images;

[0148] Methods for obtaining the central light intensity include:

[0149] The surface of the glass product is divided into n areas, where n is an integer greater than 1; the same light is emitted to the center points of the n areas through a light source, and then the luminous intensity of each center point is collected by an optical sensor and marked as the central light intensity; the center point is the most central position point in the area.

[0150] Furthermore, the method of analyzing the central light intensity to determine whether there are defects on the surface of the glass product includes:

[0151] Calculate the difference between the light intensities of each two centers in turn to obtain The light intensity difference, ; Preset the light difference threshold, and compare each light intensity difference value with the light difference threshold;

[0152] If each light intensity difference value is less than the light difference threshold, there is no defect on the surface of the glass product;

[0153] If there is a light intensity difference greater than or equal to the light difference threshold, there is a defect on the surface of the glass product;

[0154] Methods for analyzing the temperature distribution image to determine whether there are defects on the surface of the glass product include:

[0155] According to the temperature distribution image, the temperature corresponding to each position point in each area is obtained; the temperature difference between every two position points in each area is calculated respectively; a temperature difference threshold is preset, and the temperature difference of each area is compared with the temperature difference threshold;

[0156] If there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product;

[0157] If each temperature difference is less than the temperature difference threshold, count the number of location points in each area, add the temperatures of each location point in the same area in turn, and then divide it by the number of location points in the corresponding area to obtain the corresponding temperature mean of each area; calculate the difference between each two temperature means in turn to obtain The average temperature difference is calculated by comparing each average temperature difference with the temperature difference threshold; if each average temperature difference is less than the temperature difference threshold, there is no defect on the surface of the glass product; if there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product.

[0158] Furthermore, the method for identifying defect positions of glass products with surface defects includes:

[0159] Mark the area where the light intensity difference is greater than or equal to the light difference threshold as a defect area; obtain the luminous intensity of each position point in each defect area; calculate the luminous intensity difference between every two position points in the same defect area in turn, and compare each luminous intensity difference with the light difference threshold in turn; if the luminous intensity difference is greater than or equal to the light difference threshold, compare the luminous intensity of the corresponding two position points, mark the position point with large luminous intensity as the first defect point, and do not mark the position point with small luminous intensity; if the luminous intensity difference is less than the light difference threshold, do not mark the corresponding two position points;

[0160] Grayscale the temperature distribution image, collect the grayscale values ​​of Y position points, where Y is the number of all position points in the temperature distribution image; compare the grayscale values ​​of the Y position points with the grayscale value threshold respectively, mark the position points with grayscale values ​​greater than or equal to the grayscale value threshold as the second defect point, and do not mark the position points with grayscale values ​​less than the grayscale value threshold;

[0161] In the temperature distribution image, the position point marked as the first defect point according to the luminous intensity is also marked as the first defect point, and one position point can be marked as the first defect point and the second defect point at the same time; a plurality of adjacent first defect points are regarded as a first surface defect, and a plurality of adjacent second defect points are regarded as a second surface defect; a first defect point that is adjacent to both the position point and the first defect point is marked as a first contour point, and a first defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all first defect points is also marked as a first contour point; similarly, a second defect point that is adjacent to both the position point and the second defect point is marked as a second contour point, and a second defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all second defect points is also marked as a second contour point;

[0162] A rectangular coordinate system is established with any vertex of the temperature distribution image as the origin, and the coordinates corresponding to each first contour point are obtained and marked as first contour coordinates, and the coordinates corresponding to each second contour point are obtained and marked as second contour coordinates; if the coordinates of the first contour point corresponding to the first surface defect are the same as the coordinates of the second contour point corresponding to the second surface defect, the corresponding first surface defect and the second surface defect are marked as surface defects, and the corresponding first contour coordinates and the second contour coordinates are marked as contour coordinates; a scaling factor is preset, and the first contour coordinates are multiplied by the scaling factor after square root to obtain the first actual coordinates, and the second contour coordinates are multiplied by the scaling factor after square root to obtain the second actual coordinates, and the contour coordinates are multiplied by the scaling factor after square root to obtain the actual coordinates; the position of each first surface defect is identified according to the first actual coordinates, the position of each second surface defect is identified according to the second actual coordinates, and the position of each surface defect is identified according to the actual coordinates.

[0163] Furthermore, the method for identifying the defect type of a glass product having defects on the surface includes:

[0164] The first surface defect, the second surface defect and the surface defect are regarded as product surface defects; the number of position points in each product surface defect is counted, and the defect area of ​​each product surface defect is calculated; the expression of the defect area is: ; In the formula, is the defect area, is the number of position points, is the area of ​​a location point, is the scaling factor;

[0165] The area of ​​a location point is obtained by the resolution of the temperature distribution image, and the resolution of the temperature distribution image is obtained by the attributes of the temperature distribution image;

[0166] The first actual coordinate, the second actual coordinate and the actual coordinate are used as defect contour coordinates; different digital labels are set for different product surface defects and marked as defect labels; a defect label, a defect area corresponding to the defect label and a group of defect contour coordinates corresponding to the defect label are used as a group of analysis data; each group of analysis data is input into the trained type prediction model to identify the corresponding defect type; the training process of the type prediction model includes:

[0167] Pre-setting corresponding recognition results for a group of analysis data, where a is an integer greater than 1, and setting different digital labels for different recognition results; marking the digital labels of the recognition results as recognition labels, and converting the analysis data and the corresponding recognition labels into a corresponding set of feature vectors;

[0168] Each group of feature vectors is used as the input of the type prediction model. The type prediction model takes a group of predicted identification labels corresponding to each group of analysis data as output, and takes the actual identification labels corresponding to each group of analysis data as prediction targets, where the actual identification labels are the digital labels of the identification results corresponding to the analysis data collected in advance. Minimizing the sum of prediction errors of all analysis data is used as the training target. The type prediction model is trained until the sum of prediction errors reaches convergence and the training is stopped. The type prediction model is a deep neural network model. According to the obtained identification labels, the corresponding defect types are obtained.

[0169] Further, the internal data is an X-ray transmission image;

[0170] Methods for obtaining internal data include:

[0171] X-rays are emitted through an X-ray source, and then the X-ray intensity distribution passing through the glass product is recorded through a planar X-ray detector to obtain an X-ray intensity distribution image; the relative positions of the X-ray source and the planar X-ray detector are rotated in sequence, that is, each rotation is b°, b is a real number greater than 0, to obtain different irradiation angles; multiple X-ray intensity distribution images are collected from different irradiation angles, and the X-ray intensity of each pixel point in each X-ray intensity distribution image is digitally saved as a three-dimensional digital matrix with pixel point coordinates, that is, each X-ray intensity distribution image is converted into a three-dimensional digital matrix; and an X-ray transmission image is constructed according to each three-dimensional digital matrix using a reconstruction algorithm.

[0172] Furthermore, the analyzing of the internal data to determine whether there are defects inside the glass product includes: using a trained defect determination model to determine the X-ray transmission image and outputting a determination result, wherein the determination result includes whether there are defects or not;

[0173] The defect judgment model training process includes:

[0174] Collect c X-ray transmission images in advance, annotate each X-ray transmission image, and the annotations include the presence of defects and the absence of defects; convert the presence of defects and the absence of defects into digital annotations respectively; divide the annotated X-ray transmission images into a training set and a test set; use the training set to train the defect judgment model, and use the test set to test the defect judgment model; preset an error threshold, and when the mean of the prediction errors of all X-ray transmission images in the test set is less than the error threshold, output the defect judgment model; the defect judgment model is a convolutional neural network model.

[0175] Furthermore, the method for identifying the defect position of a glass product having internal defects includes:

[0176] Using the trained defect annotation model, the defect area in the X-ray transmission image is annotated with a three-dimensional rectangular frame; the eight vertex coordinates of each three-dimensional rectangular frame are obtained, and the position of each internal defect is identified according to the eight vertex coordinates of each three-dimensional rectangular frame;

[0177] The training process of the defect annotation model includes:

[0178] Pre-collect d X-ray transmission images, where d is an integer greater than 1, pre-identify the defect area in each X-ray transmission image, and annotate the defect area in the X-ray transmission image with a three-dimensional rectangular frame; divide the annotated X-ray transmission images into a training set and a test set, use 70% of the X-ray transmission images as the training set, and use 30% of the X-ray transmission images as the test set, use the training set to train the defect annotation model, and use the test set to test the defect annotation model; preset a coordinate error threshold, and when the mean of the predicted coordinate errors of all X-ray transmission images in the test set is less than the coordinate error threshold, the defect annotation model training is completed, and the defect annotation model is output; the defect annotation model is a convolutional neural network model.

[0179] Furthermore, the method for identifying the defect type of a glass product having internal defects includes:

[0180] According to the marked three-dimensional rectangular frame, the X-ray transmission image is cropped to obtain f defect images, where f is the number of three-dimensional rectangular frames; the X-ray intensities of H pixels in each defect image are collected, where H is the number of all pixels in each defect image; an intensity range value is preset, and the X-ray intensities of the H pixels in each defect image are compared with the intensity range value respectively, and the pixel point whose X-ray intensity is greater than the maximum value of the intensity range value is marked as the third defect point, and the pixel point whose X-ray intensity is less than the minimum value of the intensity range value is marked as the fourth defect point, and the pixel point whose X-ray intensity is within the intensity range is not marked;

[0181] The defect image with the third defect point is marked as a high-density image, and the image with the fourth defect point is marked as a low-density image; the grid volume integration algorithm is used to calculate the defect volume of each high-density image and low-density image; different numbers are set for high-density images and low-density images, and marked as image numbers; an image number, a defect volume and a set of vertex coordinates of a three-dimensional rectangular box are used as a set of test data, and each set of test data is input into the trained type analysis model to obtain the corresponding defect type; the training process of the type analysis model is consistent with the training process of the type prediction model, and it is also a deep neural network model.

[0182] Furthermore, the method further comprises: collecting the ambient temperature, and compensating the temperature in the temperature distribution image according to the ambient temperature;

[0183] The method for compensating the temperature in the temperature distribution image according to the ambient temperature includes:

[0184] Preset the normal temperature, subtract the normal temperature from the ambient temperature, and obtain the temperature deviation;

[0185] According to the temperature deviation and the temperature distribution image, the deviation probability corresponding to each temperature in the temperature distribution image is calculated; the expression of the deviation probability is: ; In the formula is the deviation probability corresponding to the i-th temperature in the temperature distribution image, is the temperature deviation, is the i-th temperature in the temperature distribution image, , e is a natural constant;

[0186] If the deviation probability corresponding to the temperature is greater than or equal to , then the temperature is marked as the temperature to be compensated; if the deviation probability corresponding to the temperature is less than , the temperature is not marked as the temperature to be compensated; is a real number greater than 1;

[0187] The temperature compensation amount is calculated based on the temperature to be compensated and the temperature deviation; the expression of the temperature compensation amount is: ; In the formula, is the temperature compensation corresponding to the jth temperature in the temperature distribution image, is the jth temperature in the temperature distribution image, , is the number of temperatures to be compensated, , is the preset weight coefficient;

[0188] The corresponding temperature compensation amount is subtracted from each temperature to be compensated to obtain the compensated temperature, and the temperatures to be compensated in the temperature distribution image are replaced with the corresponding compensated temperatures to complete the temperature compensation.

[0189] Example 4

[0190] See also Figure 4 As shown, the present application also provides an electronic device 500. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are run by one or more processors, the above-mentioned method for automatically identifying defects in glass products can be executed.

[0191] The method or system according to the embodiment of the present application can also be used by Figure 4 The electronic device architecture shown in FIG. Figure 4As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output 506, a hard disk 507, etc. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store a method for automatically identifying defects in glass products provided in the present application. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of an electronic device are shown.

[0192] Example 5

[0193] See also Figure 5 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a method for automatically identifying defects in glass products according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0194] In addition, according to the embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present application, for example: a method for automatically identifying defects in glass products. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0195] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0196] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically identifying defects in glass products, It is characterized in that include: Collect surface data of glass products; The surface data includes central light intensity and temperature distribution images; Methods for obtaining the central light intensity include: The surface of the glass product is divided into n areas, where n is an integer greater than 1; the same light is emitted to the center points of the n areas by a light source, and then the luminous intensity of each center point is collected by an optical sensor and marked as the center light intensity; the center point is the most central position point in the area; Analyze the surface data to determine whether there are defects on the surface of the glass product; Methods for analyzing the central light intensity to determine whether there are defects on the surface of glass products include: Calculate the difference between the light intensities of each two centers in turn to obtain The light intensity difference, ; Preset the light difference threshold, compare each light intensity difference with the light difference threshold, if each light intensity difference is less than the light difference threshold, then there is no defect on the surface of the glass product; if there is a light intensity difference greater than or equal to the light difference threshold, then there is a defect on the surface of the glass product; Methods for analyzing the temperature distribution image to determine whether there are defects on the surface of the glass product include: According to the temperature distribution image, the temperature corresponding to each position point in each area is obtained; the temperature difference between every two position points in each area is calculated respectively; a temperature difference threshold is preset, and the temperature difference of each area is compared with the temperature difference threshold; if there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product; if each temperature difference is less than the temperature difference threshold, the number of position points in each area is counted, the temperature of each position point in the same area is added in turn, and then divided by the number of position points in the corresponding area to obtain the temperature mean corresponding to each area; the difference between every two temperature means is calculated in turn to obtain temperature difference; then compare each temperature difference with the temperature difference threshold; if each temperature difference is less than the temperature difference threshold, there is no defect on the surface of the glass product; if there is a temperature difference greater than or equal to the temperature difference threshold, there is a defect on the surface of the glass product; Identify the defect type and location of glass products with surface defects; Methods for identifying defect locations of glass products with surface defects include: Mark the area where the light intensity difference is greater than or equal to the light difference threshold as a defect area; obtain the luminous intensity of each position point in each defect area; calculate the luminous intensity difference between every two position points in the same defect area in turn, and compare each luminous intensity difference with the light difference threshold in turn; if the luminous intensity difference is greater than or equal to the light difference threshold, compare the luminous intensity of the corresponding two position points, mark the position point with large luminous intensity as the first defect point, and do not mark the position point with small luminous intensity; if the luminous intensity difference is less than the light difference threshold, do not mark the corresponding two position points; Grayscale the temperature distribution image, collect the grayscale values ​​of Y position points, where Y is the number of all position points in the temperature distribution image; compare the grayscale values ​​of the Y position points with the grayscale value threshold respectively, mark the position points with grayscale values ​​greater than or equal to the grayscale value threshold as the second defect point, and do not mark the position points with grayscale values ​​less than the grayscale value threshold; In the temperature distribution image, the position point marked as the first defect point according to the luminous intensity is also marked as the first defect point, and one position point can be marked as the first defect point and the second defect point at the same time; a plurality of adjacent first defect points are regarded as a first surface defect, and a plurality of adjacent second defect points are regarded as a second surface defect; a first defect point that is adjacent to both the position point and the first defect point is marked as a first contour point, and a first defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all first defect points is also marked as a first contour point; similarly, a second defect point that is adjacent to both the position point and the second defect point is marked as a second contour point, and a second defect point that is located at the outermost periphery of the temperature distribution image and whose adjacent position points are all second defect points is also marked as a second contour point; Establish a rectangular coordinate system with any vertex of the temperature distribution image as the origin, obtain the coordinates corresponding to each first contour point, and mark them as first contour coordinates, obtain the coordinates corresponding to each second contour point, and mark them as second contour coordinates; if the coordinates of the first contour point corresponding to the first surface defect are the same as the coordinates of the second contour point corresponding to the second surface defect, mark the corresponding first surface defect and the second surface defect as surface defects, and mark the corresponding first contour coordinates and the second contour coordinates as contour coordinates; preset a scaling factor, multiply the first contour coordinates by the scaling factor after square root, obtain the first actual coordinates, multiply the second contour coordinates by the scaling factor after square root, obtain the second actual coordinates, multiply the contour coordinates by the scaling factor after square root, obtain the actual coordinates; identify the position of each first surface defect according to the first actual coordinates, identify the position of each second surface defect according to the second actual coordinates, and identify the position of each surface defect according to the actual coordinates; Methods for identifying defect types of glass products with surface defects include: The first surface defect, the second surface defect and the surface defect are regarded as product surface defects; the number of position points in each product surface defect is counted, and the defect area of ​​each product surface defect is calculated; the expression of the defect area is: ; In the formula, is the defect area, is the number of position points, is the area of ​​a location point, is the scaling factor; The area of ​​a location point is obtained by the resolution of the temperature distribution image, and the resolution of the temperature distribution image is obtained by the attributes of the temperature distribution image; The first actual coordinate, the second actual coordinate and the actual coordinate are used as defect contour coordinates; different digital labels are set for different product surface defects and marked as defect labels; a defect label, a defect area corresponding to the defect label and a group of defect contour coordinates corresponding to the defect label are used as a group of analysis data; each group of analysis data is input into the trained type prediction model to identify the corresponding defect type; the training process of the type prediction model includes: Pre-setting corresponding recognition results for a group of analysis data, where a is an integer greater than 1, and setting different digital labels for different recognition results; marking the digital labels of the recognition results as recognition labels, and converting the analysis data and the corresponding recognition labels into a corresponding set of feature vectors; Each group of feature vectors is used as the input of the type prediction model. The type prediction model uses a group of predicted identification labels corresponding to each group of analysis data as output, and uses the actual identification labels corresponding to each group of analysis data as the prediction target. The actual identification labels are pre-collected digital labels of the identification results corresponding to the analysis data; minimizing the sum of the prediction errors of all analysis data is used as the training target; the type prediction model is trained until the sum of the prediction errors reaches convergence and the training is stopped; the type prediction model is a deep neural network model; according to the obtained identification labels, the corresponding defect type is obtained; Collecting internal data of the glass product; the internal data is an X-ray transmission image; Methods for obtaining internal data include: X-rays are emitted through an X-ray source, and then the X-ray intensity distribution passing through the glass product is recorded through a planar X-ray detector to obtain an X-ray intensity distribution image; the relative positions of the X-ray source and the planar X-ray detector are rotated in sequence, that is, each rotation is b°, where b is a real number greater than 0, to obtain different irradiation angles; multiple X-ray intensity distribution images are collected from different irradiation angles, and the X-ray intensity of each pixel point in each X-ray intensity distribution image is digitally saved as a three-dimensional digital matrix with pixel point coordinates, that is, each X-ray intensity distribution image is converted into a three-dimensional digital matrix; and an X-ray transmission image is constructed using a reconstruction algorithm according to each three-dimensional digital matrix; Analyze internal data to determine whether there are defects inside the glass products; The analyzing of the internal data to determine whether there are defects inside the glass product includes: using a trained defect judgment model to judge the X-ray transmission image and outputting a judgment result, wherein the judgment result includes whether there are defects or not; The defect judgment model training process includes: Collect c X-ray transmission images in advance, annotate each X-ray transmission image, and the annotations include the presence of defects and the absence of defects; convert the presence of defects and the absence of defects into digital annotations respectively; divide the annotated X-ray transmission images into a training set and a test set; use the training set to train the defect judgment model, and use the test set to test the defect judgment model; preset an error threshold, and when the mean of the prediction errors of all X-ray transmission images in the test set is less than the error threshold, output the defect judgment model; the defect judgment model is a convolutional neural network model; Identify the defect type and location of glass products with internal defects; Methods for identifying defect locations of glass products with internal defects include: Using the trained defect annotation model, the defect area in the X-ray transmission image is annotated with a three-dimensional rectangular frame; the eight vertex coordinates of each three-dimensional rectangular frame are obtained, and the position of each internal defect is identified according to the eight vertex coordinates of each three-dimensional rectangular frame; The training process of the defect annotation model includes: Pre-collect d X-ray transmission images, where d is an integer greater than 1, pre-identify the defect area in each X-ray transmission image, and annotate the defect area in the X-ray transmission image with a three-dimensional rectangular frame; divide the annotated X-ray transmission images into a training set and a test set, use 70% of the X-ray transmission images as the training set, and use 30% of the X-ray transmission images as the test set, use the training set to train the defect annotation model, and use the test set to test the defect annotation model; preset a coordinate error threshold, and when the mean of the predicted coordinate errors of all X-ray transmission images in the test set is less than the coordinate error threshold, the defect annotation model training is completed, and the defect annotation model is output; the defect annotation model is a convolutional neural network model; Methods for identifying defect types of glass products with internal defects include: According to the marked three-dimensional rectangular frame, the X-ray transmission image is cropped to obtain f defect images, where f is the number of three-dimensional rectangular frames; the X-ray intensities of H pixels in each defect image are collected, where H is the number of all pixels in each defect image; an intensity range value is preset, and the X-ray intensities of the H pixels in each defect image are compared with the intensity range value respectively, and the pixel point whose X-ray intensity is greater than the maximum value of the intensity range value is marked as the third defect point, and the pixel point whose X-ray intensity is less than the minimum value of the intensity range value is marked as the fourth defect point, and the pixel point whose X-ray intensity is within the intensity range is not marked; The defect image with the third defect point is marked as a high-density image, and the image with the fourth defect point is marked as a low-density image; the grid volume integration algorithm is used to calculate the defect volume of each high-density image and low-density image; different numbers are set for high-density images and low-density images, and marked as image numbers; an image number, a defect volume and a set of vertex coordinates of a three-dimensional rectangular box are used as a set of test data, and each set of test data is input into the trained type analysis model to obtain the corresponding defect type; the training process of the type analysis model is consistent with the training process of the type prediction model, and it is also a deep neural network model.

2. A method for automatically identifying defects in glass products according to claim 1, It is characterized in that The method further comprises: collecting the ambient temperature, and compensating the temperature in the temperature distribution image according to the ambient temperature; The method for compensating the temperature in the temperature distribution image according to the ambient temperature includes: Preset the normal temperature, subtract the normal temperature from the ambient temperature, and obtain the temperature deviation; According to the temperature deviation and the temperature distribution image, the deviation probability corresponding to each temperature in the temperature distribution image is calculated; the expression of the deviation probability is: ; In the formula is the deviation probability corresponding to the i-th temperature in the temperature distribution image, is the temperature deviation, is the i-th temperature in the temperature distribution image, , e is a natural constant; If the deviation probability corresponding to the temperature is greater than or equal to , then the temperature is marked as the temperature to be compensated; if the deviation probability corresponding to the temperature is less than , the temperature is not marked as the temperature to be compensated; is a real number greater than 1; The temperature compensation amount is calculated based on the temperature to be compensated and the temperature deviation; the expression of the temperature compensation amount is: ; In the formula, is the temperature compensation corresponding to the jth temperature in the temperature distribution image, is the jth temperature in the temperature distribution image, , is the number of temperatures to be compensated, , is the preset weight coefficient; The corresponding temperature compensation amount is subtracted from each temperature to be compensated to obtain the compensated temperature, and the temperatures to be compensated in the temperature distribution image are replaced with the corresponding compensated temperatures to complete the temperature compensation.

3. A glass product defect automatic identification system, used to implement the glass product defect automatic identification method according to any one of claims 1 to 2, It is characterized in that include: A surface data acquisition module is used to collect surface data of glass products; The surface defect judgment module is used to analyze the surface data and judge whether there are defects on the surface of the glass product; A surface defect recognition module is used to identify the defect type and defect location of glass products with surface defects; Internal data collection module, used to collect internal data of glass products; Internal defect judgment module, used to analyze internal data and judge whether there are defects inside the glass product; The internal defect recognition module is used to identify the defect type and defect location of glass products with internal defects.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for automatically identifying defects in glass products according to any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method for automatically identifying defects in glass products according to any one of claims 1 to 2 is implemented.