An image recognition-based building glass safety evaluation system

Abstract

The application belongs to the technical field of glass detection, and specifically discloses a building glass safety evaluation system based on image recognition, which is characterized by the following: a plurality of multi-modal image acquisition modules are arranged along a production line conveyor belt to collect glass plate image data from multiple perspectives; a three-dimensional model reconstruction module is used to construct a glass plate digital model with an accurate three-dimensional coordinate system; a defect identification and classification module extracts quantitative features from the image and outputs an identification result; and a sorting execution module drives a sorting mechanism to perform automatic sorting according to the verified result. The application realizes the leap from two-dimensional identification to three-dimensional spatial accurate measurement and positioning, and improves the comprehensiveness of detection, classification accuracy and system reliability.

Description

Technical Field

[0001] This invention belongs to the field of glass inspection technology, and specifically relates to a building glass safety assessment system based on image recognition. Background Technology

[0002] As a crucial component of modern building envelopes and load-bearing structures, architectural glass's internal and surface defects directly impact its mechanical properties and safety. Currently, automated quality inspection methods in the industry primarily rely on machine vision technology, using industrial cameras to capture image information of the glass and employing image processing algorithms or artificial intelligence models for defect identification and classification.

[0003] Current detection and analysis processes typically acquire images from a single fixed perspective or a combination of different perspectives. However, the lack of a three-dimensional reference for image information makes it impossible to spatially align defect information acquired under different optical modes, and it is also impossible to accurately calculate the actual physical size of the defect or its specific location along the thickness of the glass. For architectural glass, the embedment depth of scratches or cracks within the glass and their relative distance from edges or stress areas are key parameters for assessing their impact on the safety performance of the glass structure. Existing detection technologies cannot provide location parameters for scratches or cracks, resulting in a lack of accurate spatial data to support safety assessments. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an image recognition-based building glass safety assessment system.

[0005] The present invention provides an image recognition-based building glass safety assessment system, including a production line conveyor belt connected to a glass production line, a number of detection stations arranged sequentially along the conveying direction of the production line conveyor belt, a sorting mechanism arranged on one side of the tail of the production line conveyor belt, and a display panel arranged on one side of the sorting mechanism. The inspection station is equipped with a multimodal image acquisition module, which is used to sequentially acquire multimodal image data of the building glass panel from different perspectives and different optical modes. The display panel integrates a 3D model reconstruction module, a defect identification and classification module, and a sorting execution module. The 3D model reconstruction module is connected to the multimodal image acquisition module, and is used to receive the multimodal image data, construct a 3D model of the building glass panel, and establish a 3D coordinate system for spatial measurement in the 3D model; The defect identification and classification module is connected to the multimodal image acquisition module and the three-dimensional model reconstruction module respectively. It is used to extract the quantitative features of defects from the image data, and classify linear surface defects into scratches or cracks according to the preset weighted decision model, and output the identification results including defect type, quantitative features and three-dimensional position. The sorting execution module is communicatively connected to the defect identification and classification module, and drives the sorting mechanism to move the building glass panels to different areas based on the defect identification results.

[0006] A further embodiment is that the multimodal image acquisition module includes an acquisition control unit and multiple detection stations arranged sequentially along the production line conveyor belt. The detection stations include a transmitted light imaging unit, a reflected light imaging unit, and a polarization stress imaging unit. The transmitted light imaging unit includes a uniform backlight source and a first industrial camera located on the opposite side of the building glass panel, used to acquire transmitted light images of the building glass panel to detect bubbles and impurities. The reflected light imaging unit includes a low-angle ring LED light source and a second industrial camera. The ring light source is tilted to illuminate the building glass panel and is used to acquire reflected light images to detect surface scratches, cracks and stains. The polarization stress imaging unit includes a polarizer array and a third industrial camera. The polarizer array is set in front of the lens of the third industrial camera to acquire polarization stress images of the stress distribution and concentration state inside the building glass panel based on different polarization angles. The acquisition and control unit is connected to the position sensor of the production line conveyor belt and is used to control the camera, light source, parameters and viewing angle of the transmitted light imaging unit, reflected light imaging unit and polarized stress imaging unit when the building glass panel arrives at the inspection station so as to acquire multimodal image data in sequence.

[0007] A further embodiment is that the 3D model reconstruction module includes: The point cloud generation unit is used to receive image data from multiple perspectives and calculate and generate high-density three-dimensional point cloud data representing the key structures of the surface and interior of the building glass panel through feature point matching and triangulation algorithms. A surface meshing unit, connected to the point cloud generation unit, is used to convert the three-dimensional point cloud data into a continuous triangular mesh surface model through a Poisson reconstruction algorithm. The coordinate system construction and texture mapping unit, connected to the surface meshing unit, is used to define a three-dimensional spatial coordinate system based on the physical characteristics of the architectural glass panel in the triangular mesh surface model, and to map the original image texture onto the model surface to form a measurable three-dimensional visual digital model with realistic texture.

[0008] A further embodiment is that the defect identification and classification module includes: The internal defect analysis unit, connected to the transmitted light imaging unit, is used to receive and process transmitted light images, identify bubbles and impurities through adaptive threshold segmentation and morphological analysis, and calculate their number, size, area ratio and spatial distribution density in the three-dimensional model. A surface defect extraction unit, connected to the reflected light imaging unit, is used to receive and process reflected light images, extract potential linear defect regions using a directional edge detection algorithm, and obtain a sub-pixel precision contour coordinate sequence of the linear defect regions. The feature quantization and classification unit, connected to the surface defect extraction unit, is used to fit the linear defect contour coordinate sequence into a feature line, and calculate the local width, depth, shape smoothness and width fluctuation amplitude along the feature line, and classify the linear defect into scratches or cracks based on a weighted decision model.

[0009] A further proposed solution is that the weighted decision model is as follows: ; in, For normalized shape smoothness eigenvalues, For the normalized width eigenvalues, For normalized depth feature values, The normalized width fluctuation amplitude characteristic value; , , , These are the weight coefficients for the corresponding features, satisfying... ,and Configured as a negative value; The feature quantization and classification unit presets a decision threshold. When the calculated A value is less than or equal to the decision threshold, the defect is determined to be a scratch. When the value is greater than the judgment threshold, the defect is determined to be a crack.

[0010] A further embodiment is that the display panel integrates a conflict detection module, which is connected to the multimodal image acquisition module, the defect identification and classification module, and the sorting execution module, respectively. This module is used to verify the identification results of the defect identification and classification module and send the final verified correct result to the sorting execution module.

[0011] A further approach is to configure the collision detection module to perform the following steps: The system receives the identification result output by the defect identification and classification module, and extracts the three-dimensional surface geometry data and texture data of the corresponding coordinate region from the three-dimensional model reconstruction module based on the multimodal image data corresponding to the identification result and the three-dimensional position; the multimodal image data includes at least the transmitted light image, reflected light image and polarization stress image of the same coordinate region; For identification results classified as cracks, verify whether there are local stress concentration patterns in the corresponding polarized stress image that match the mechanical characteristics of cracks; and for identification results classified as internal defects, verify whether the three-dimensional surface geometric data presents a continuous and flat shape. If yes, the output of the defect identification and classification module is determined to be correct; otherwise, the output of the defect identification and classification module is determined to be incorrect. The multimodal image data corresponding to the erroneous results are processed, and then the defects are classified again through the defect identification and classification module. Repeat the above steps until the defect identification and classification module outputs the correct classification result.

[0012] A further approach is that the verification steps for the local stress concentration pattern include: Based on the reflected light image, sub-pixel contours identified as crack defects are extracted in the region corresponding to the three-dimensional position; the major axis direction of the minimum bounding rectangle of the sub-pixel contour is defined as the reference direction of the crack. In the polarization stress image, an analysis region is defined with the three-dimensional position as the center; the stress gradient magnitude and direction of the pixels in the analysis region are calculated; pixels with gradient magnitudes exceeding a first predetermined threshold are identified, and it is determined whether a spatially continuous pixel cluster is formed. If a pixel cluster is formed, calculate the average gradient direction of the pixels within the pixel cluster and the angle between the average gradient direction and the reference orientation; if the angle is less than a second predetermined threshold, it is determined to be a local stress concentration pattern.

[0013] A further solution involves verifying whether the three-dimensional surface geometry data exhibits a continuous and flat shape, including the following steps: Based on the three-dimensional surface geometry data, calculate the local surface curvature within a predetermined radius centered on the three-dimensional position; If the local surface curvature is less than the curvature threshold, the three-dimensional surface geometry is determined to be a continuous and flat shape. If the local surface curvature is greater than or equal to the curvature threshold, it is determined that the three-dimensional surface geometry does not present a continuous flat shape.

[0014] A further solution is that the conflict detection module also includes an error case database and a data optimization model. The error case database is used to store the number of error type-effective handling method pairs recorded by the conflict detection module and ultimately verified by the defect identification and classification module. Each error type-effective processing method pair includes different error types output by the defect identification and classification module, as well as the processing method of the original multimodal image data corresponding to different error types; The data optimization model outputs effective processing methods based on error types; The process of constructing the data optimization model is as follows: A large number of error type-handling method pairs are obtained and labeled by human experts; after labeling, the error type-handling method pairs are input into the neural network unit for iterative training; in order to obtain a data optimization model based on the error type output for the preprocessing method of the original multimodal image data.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention acquires data from different perspectives and optical modes using a multimodal image acquisition module arranged along the production line conveyor belt. The data is then processed by a 3D model reconstruction module to finally construct a 3D model of the architectural glass panel. This allows the system to obtain the 3D geometric information of the object's surface, enabling the detected defects to be uniformly integrated into the same 3D digital model. The system outputs identification results that include defect type, quantitative features, and 3D location, providing data support for accurate safety assessment of architectural glass based on the 3D location, size, and spatial relationship of defects.

[0016] The multimodal image acquisition module of this invention integrates a transmitted light imaging unit, a reflected light imaging unit, and a polarized stress imaging unit. Transmitted light imaging utilizes uniform backlighting to penetrate the glass, clearly revealing internal volumetric defects such as bubbles and impurities. Reflected light imaging employs a low-angle ring light source with tilted illumination to acquire surface features such as scratches and cracks. Polarized stress imaging analyzes changes in light intensity at different polarization angles to acquire the stress distribution and concentration areas within the glass. Through multimodal collaboration, the correlation between defects can be represented, thereby providing a comprehensive assessment of the overall safety of the glass. This module is suitable for building curtain wall glass and load-bearing glass with high safety requirements, improving the completeness and reliability of the inspection.

[0017] The defect identification and classification module of this invention extracts sub-pixel precision linear defect contours from reflected light images, and then systematically calculates four quantitative features along the contour lines: local width, depth, shape smoothness, and width fluctuation amplitude. By fusing these features and assigning weights, stable and high-precision automatic classification of scratches and cracks is achieved. Compared to methods that rely entirely on black-box deep neural networks for end-to-end image classification, this solution significantly reduces the dependence on massive, balanced labeled data and improves the ability to classify cracks in industrial scenarios.

[0018] The collision detection module of this invention receives the preliminary results from the defect identification and classification module and simultaneously retrieves the corresponding original multimodal image data and local geometric and texture data extracted from the 3D model for logical consistency cross-validation. This effectively identifies and corrects false alarms and misclassifications in the defect identification and classification module. By introducing polarization stress and 3D geometry—two types of information independent of traditional RGB images—for cross-validation, the reliability and confidence of the overall output are significantly improved. This reduces the risk of missed detections of cracks and scratches that might otherwise be found in qualified products, playing a crucial role in ensuring the safety of architectural glass and optimizing production and sorting costs. The collision detection module of this invention also integrates an error case pair library and a data optimization model, forming a continuously optimizing intelligent learning loop. Specifically, during operation, the system automatically collects all cases discovered by collision detection and successfully corrected through reclassification. Each case is constructed as a structured error type-effective handling method pair and stored in the library. The data optimization model, through training with a large number of such pairs, learns the mapping relationship from error type to optimal image preprocessing, enabling the entire evaluation system to learn from practice and improving the system's accuracy in defect identification. Attached Figure Description

[0019] The following figures are for illustrative purposes only and are not intended to limit the scope of the invention, wherein: Figure 1 : Schematic diagram of the connection structure of the present invention; Figure 2 : Block diagram of the evaluation system of this invention; In the diagram: 1. Production line conveyor belt; 2. Sorting mechanism; 3. Architectural glass panel; 4. First inspection station; 5. Second inspection station; 6. Third inspection station; 7. Display panel; 8. Multimodal image acquisition module; 9. 3D model reconstruction module; 10. Defect recognition and classification module; 11. Conflict detection module; 12. Point cloud generation unit; 13. Surface meshing unit; 14. Coordinate system construction and texture mapping unit; 15. Internal defect analysis unit; 16. Surface defect extraction unit; 17. Feature quantization and classification unit; 18. Weighted decision model; 19. Error case database; 20. Data optimization model; 21. Transmitted light imaging unit; 22. Reflected light imaging unit; 23. Polarization stress imaging unit; 24. Acquisition and control unit; 25. Sorting execution module. Detailed Implementation

[0020] To make the objectives, technical solutions, design methods, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0021] like Figure 1 and Figure 2 As shown, this invention provides an image recognition-based safety assessment system for architectural glass. The system includes a production line conveyor belt 1 connected to a glass production line. Several inspection stations are sequentially arranged along the conveyor belt 1 to perform multimodal inspections on the architectural glass panels 3 conveyed on the production line conveyor belt 1. A sorting mechanism 2 for performing sorting operations is installed on one side of the tail end of the production line conveyor belt 1. A display panel 7 is provided on one side of the sorting mechanism 2. During system operation, the architectural glass panels 3 first enter the inspection area along with the production line conveyor belt 1. The multimodal image acquisition module 8 inside each inspection station sequentially and in order acquires multimodal image data of the architectural glass panels 3 from multiple preset different perspectives and different optical modes such as transmission, reflection, and polarization. Multimodal image data is transmitted in real time to the 3D model reconstruction module 9 integrated within the display panel 7. The 3D model reconstruction module 9 utilizes the multi-view solid geometry principle of computer vision to fuse two-dimensional image sequences, constructing a 3D digital model of the architectural glass panel 3 with precise dimensional information. A 3D coordinate system based on the boundary of the architectural glass panel 3 is established within the 3D digital model, providing a reference for all spatial measurements. The defect identification and classification module 10 within the display panel 7 simultaneously receives image data from the multimodal image acquisition module 8 and model data from the 3D model reconstruction module 9. It extracts the geometric, textural, and other quantitative features of defects from the image data. Then, based on a preset weighted decision model 18, it intelligently classifies similar linear surface defects into scratches or cracks, and finally outputs the identification result. This identification result includes the defect type, detailed quantitative features, and its precise 3D location in the 3D model. The sorting execution module 25 within the display panel 7 generates control commands based on the identification result and drives the sorting mechanism 2 to transfer the architectural glass panel 3 to the corresponding qualified area, rework area, or scrap area. For example, when the system identifies a crack on a building glass panel 3 that exceeds the safety standard in depth and is located in the stress zone of future installation, the sorting execution module will instruct the sorting mechanism 2 to send it to the scrap area; if only some small, shallow scratches are detected, the sorting mechanism 2 will be instructed to send them to the qualified area, thereby realizing fully automated online detection and grading.

[0022] In this embodiment, the multimodal image acquisition module 8 consists of an acquisition control unit 24 and multiple physical inspection stations arranged linearly along the production line conveyor belt 1. Specifically, it includes a first inspection station 4, a second inspection station 5, and a third inspection station 6 arranged sequentially along the production line conveyor belt. Each of the first inspection station 4, second inspection station 5, and third inspection station 6 is externally enclosed in a sealed enclosure to isolate it from external natural light. The first inspection station 4 contains a transmitted light imaging unit 21; the second inspection station 5 contains a reflected light imaging unit 22; and the third inspection station 6 contains a polarized stress imaging unit 23. The transmitted light imaging unit 21 includes a backlight source emitting uniform light and a first industrial camera located opposite the architectural glass panel 3. Light penetrates the architectural glass panel 3, and the first industrial camera receives the transmitted light, thus clearly displaying internal defects such as bubbles and stones in the image. The reflected light imaging unit 22 includes a low-angle-mounted ring LED light source and a second industrial camera. The ring LED light source illuminates the surface of the architectural glass panel 3 at an oblique angle, and the second industrial camera receives the reflected light from the surface. Since low-angle illumination can significantly enhance the contrast of surface scratches, microcracks, and other morphological defects, images of surface scratches and cracks can be acquired through the second industrial camera. The polarization stress imaging unit 23 is a polarizer array with a controllable angle installed in front of the lens of the third industrial camera. By rotating the polarizer and acquiring image information, based on the principle of photoelasticity, the stress distribution state inside the architectural glass panel 3 is inverted to detect stress concentration areas. The acquisition control unit 24 is connected to the photoelectric sensor installed on the production line conveyor belt 1. When the photoelectric sensor detects that the architectural glass panel 3 has reached the predetermined station, the acquisition control unit 24 immediately triggers the transmitted light imaging unit 21, the reflected light imaging unit 22, and the polarization stress imaging unit 23 in sequence according to a preset timing, and synchronously adjusts the light source intensity, camera exposure parameters, and shooting angle of each unit to ensure that high-quality, complementary multimodal image data is acquired in sequence.

[0023] In the above, the 3D model reconstruction module 9 includes a point cloud generation unit 12, a surface meshing unit 13, and a coordinate system construction and texture mapping unit 14. The point cloud generation unit 12 receives image data from the multimodal image acquisition module 8, finds the correspondence of the same physical feature point in multiple images using a feature point matching algorithm, and calculates the precise coordinates of the feature point in 3D space using a triangulation algorithm, combining pre-calibrated camera parameters to generate 3D point cloud data representing the key structures of the surface and interior of the architectural glass panel 3. The surface meshing unit 13 receives the 3D point cloud data output by the point cloud generation unit 12 and transforms the discrete 3D point cloud into a continuous, closed, and smooth triangular mesh surface model. The coordinate system construction and texture mapping unit 14 receives the triangular mesh surface model generated by the surface meshing unit 13 and defines a 3D spatial coordinate system on the triangular mesh surface model. In this embodiment, a corner point of the architectural glass panel 3 is selected as the origin, two adjacent edges are used as the X-axis and Y-axis, and the normal direction is used as the Z-axis. The original high-resolution image acquired by the multimodal image acquisition module 8 is used as a texture and accurately mapped onto the corresponding triangular mesh surface according to the camera projection model, forming a three-dimensional visual digital model that contains both precise geometric dimensions and appearance texture.

[0024] The internal defect analysis unit 15 of the defect identification and classification module 10 is directly connected to the output of the transmitted light imaging unit 21. For the received transmitted light image, an adaptive threshold segmentation algorithm is used to binarize the image, separating suspected defect areas. Morphological analysis is then used to remove noise and smooth region boundaries, thereby accurately identifying internal defects such as bubbles and impurities. Furthermore, the number, equivalent diameter, and area percentage of each defect are calculated. Finally, this two-dimensional information is mapped to the three-dimensional model generated by the three-dimensional model reconstruction module 9 to calculate its spatial distribution density. The surface defect extraction unit 16 is connected to the reflected light imaging unit 22. An edge detection algorithm is used to extract the linearly distributed defect region contours in the image, and sub-pixel interpolation technology is used to obtain the coordinate sequence of these contours. The feature quantization and classification unit 17 receives the sub-pixel contour coordinate sequence output by the surface defect extraction unit 16, fits the discrete coordinate points into a smooth feature line, samples along the feature line at fixed intervals, and calculates the local width, local depth, shape smoothness, and width fluctuation amplitude along the feature line direction at each sampling point. These four normalized feature values ​​are then input into the pre-trained weighted decision model 18 for comprehensive scoring and classification. Specifically, the weighted decision model 18 is a discriminant function based on linear weighting, and its mathematical expression is: In this model, For normalized shape smoothness eigenvalues, For the normalized width eigenvalues, For normalized depth feature values, The normalized width fluctuation amplitude characteristic value; , , , These are the weight coefficients for the corresponding features. Since cracks have greater depth and more irregular propagation paths, the depth feature... weight Maximum; width feature The differentiation is relatively weak, and the weight is low. Smaller, smoother shape It helps to distinguish the twists and turns of natural cracks from the relative straightness of mechanical scratches. Between and Between; and the width fluctuation range It is a positively correlated feature for typical scratches, but a negatively correlated feature for cases that we want to classify as cracks; therefore, its weight... It is configured as a negative value. A decision threshold T is preset within the feature quantization and classification unit 17. When the comprehensive evaluation value A calculated for a linear defect is less than or equal to T, the defect identification and classification module 10 determines the defect as a scratch; when the value A is greater than T, it is determined as a crack. For example, a linear defect with a relatively deep depth, uneven edges, but relatively uniform width will be classified as a scratch due to its depth weight. High and width fluctuation contribution A negative value results in a higher A value, thus correctly classifying it as a crack.

[0025] To improve system reliability, a collision detection module 11 is integrated within the display panel 7. This collision detection module 11 establishes communication links with the multimodal image acquisition module 8, the defect identification and classification module 10, and the sorting execution module. It performs secondary verification on the preliminary identification results output by the defect identification and classification module 10. The collision detection module 11 is triggered when it receives an information packet containing the defect type and three-dimensional location from the defect identification and classification module 10. Cross-validation using multi-source data ensures that the final action commands driving the sorting execution module are highly reliable.

[0026] The execution logic of conflict detection module 11 is as follows: The conflict detection module 11 receives the recognition result from the defect identification and classification module 10. Based on the precise three-dimensional location provided in the result, it requests the original multimodal image data corresponding to the same coordinate region from the multimodal image acquisition module 8, and simultaneously requests the extraction of three-dimensional surface geometry and texture data of the region surrounding the coordinate point from the three-dimensional model reconstruction module 9. Next, the conflict detection module 11 applies preset verification rules: for recognition results classified as cracks, it checks whether there is a local stress concentration pattern in the corresponding polarized stress image near the three-dimensional location that matches the mechanical characteristics of a crack; for recognition results classified as internal defects (such as bubbles), it checks whether the three-dimensional surface geometry data extracted from the three-dimensional model presents a continuous and flat shape at the location to eliminate interference caused by surface attachments (such as water stains and oil stains) under transmitted light. If the verification passes, the original output of the defect identification and classification module 10 is determined to be correct, and the result is forwarded to the sorting execution module 24. If the verification fails, the output is deemed incorrect. In this case, the conflict detection module 11 applies a series of preprocessing operations (e.g., contrast stretching, directional filtering, or noise suppression) to the original multimodal image data that caused the error. Then, the processed image data is sent back to the defect identification and classification module 10, requesting it to reclassify the defects. This continues until the defect identification and classification module 10 outputs a classification result that passes all physical consistency checks and is ultimately confirmed as correct.

[0027] The steps for verifying local stress concentration patterns described above are as follows: Based on the three-dimensional position provided by the defect identification and classification module 10, the conflict detection module 11 locates the corresponding reflected light image area and accurately extracts the sub-pixel contours identified as cracks in that area.

[0028] Calculate the minimum bounding rectangle of the subpixel profile, and mathematically define the major axis direction of this rectangle as the reference direction of the crack.

[0029] The collision detection module 11 switches to the polarization stress image at the same location and delineates a rectangular analysis area centered on the given three-dimensional position coordinates.

[0030] The stress gradient of each pixel in the X and Y directions is calculated using the image gradient operator, thereby obtaining the gradient magnitude and gradient direction of each pixel.

[0031] Set a first predetermined threshold and identify all high-gradient pixels whose gradient magnitude exceeds this threshold. Spatial connectivity analysis is performed on these high-gradient pixels to determine whether they can form one or more spatially contiguous pixel clusters. If a valid pixel cluster cannot be formed, the verification fails; if a significant pixel cluster is formed, the average gradient direction of all pixels within that cluster is calculated.

[0032] Calculate the angle between this average gradient direction and the reference direction.

[0033] A second predetermined threshold is set. If the calculated included angle is less than the second predetermined threshold, the conflict detection module 11 determines that there is a local stress concentration pattern in the polarization stress image that is highly consistent with the crack direction, thereby supporting the crack classification conclusion and passing the verification.

[0034] The method for verifying the flatness of a three-dimensional surface described above is as follows: When the conflict detection module 11 needs to verify a result classified as an internal defect, it extracts local three-dimensional surface geometric data from the data interface provided by the three-dimensional model reconstruction module 9, within a spherical space with radius R centered on the three-dimensional position coordinates of the defect. The collision detection module 11 calculates the surface curvature of the triangular mesh in the local area. The system has a preset curvature threshold. If the calculated local surface curvature value is less than the curvature threshold, the collision detection module 11 determines that the three-dimensional surface geometry data at this location presents a continuous and flat shape. If the local surface curvature value is greater than or equal to the curvature threshold, the system determines that the surface is uneven, which may indicate the presence of surface stains, coating peeling, or minor bumps. The original internal defect classification may be incorrect, and the verification fails.

[0035] To enhance the intelligence and adaptability of the system during long-term operation, the conflict detection module 11 also includes an error case pair library 19 and a data optimization model 20. The error case pair library 19 stores experiential knowledge generated during system operation. Specifically, whenever the conflict detection module 11 detects an initial misclassification by the defect identification and classification module 10, and after image processing guided by itself, is successfully corrected by the defect identification and classification module 10, the entire process is encapsulated as an "error type-effective handling method" pair. For example, the error type might be "a shallow scratch misjudged as a crack against a highly reflective background," while the corresponding effective handling method is "applying homomorphic filtering to the reflected light image to compress the dynamic range, and then re-extracting features." The data optimization model 20 is a trained neural network. Its construction process involves: domain experts cleaning and standardizing a large number of historically accumulated "error type-handling method" pairs to form a high-quality supervised learning sample set. These samples are then input into a neural network unit for iterative training, and the network parameters are adjusted through a backpropagation algorithm. After training, a data optimization model 20 is obtained. When a new error type appears, the data optimization model 20 can quickly infer and output one or a set of most likely successful image data processing strategy suggestions based on the characteristics of the error, which can be called by the conflict detection module 11, thereby significantly accelerating the error correction cycle and enabling the system to handle previously unseen but similar error patterns.

[0036] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A building glass safety assessment system based on image recognition, comprising a production line conveyor belt (1) connected to a glass production line, characterized in that, Several inspection stations are arranged sequentially along the conveying direction of the production line conveyor belt (1). A sorting mechanism (2) is arranged on one side of the tail of the production line conveyor belt (1), and a display panel (7) is arranged on one side of the sorting mechanism (2). The detection station is equipped with a multimodal image acquisition module (8) for sequentially acquiring multimodal image data of the building glass panel (3) from different perspectives and different optical modes. The display panel (7) integrates a three-dimensional model reconstruction module (9), a defect identification and classification module (10), and a sorting execution module (25). The three-dimensional model reconstruction module (9) is connected to the multimodal image acquisition module (8) to receive the multimodal image data, construct the three-dimensional model of the building glass panel (3), and establish a three-dimensional coordinate system for spatial measurement in the three-dimensional model; The defect identification and classification module (10) is connected to the multimodal image acquisition module (8) and the three-dimensional model reconstruction module (9) respectively. It is used to extract the quantitative features of defects from the image data, and classify linear surface defects into scratches or cracks according to the preset weighted decision model, and output the identification results including defect type, quantitative features and three-dimensional position. The sorting execution module (25) is connected to the defect identification and classification module (10) and drives the sorting mechanism (2) to move the building glass panel (3) to different areas based on the defect identification result.

2. The building glass safety assessment system based on image recognition according to claim 1, characterized in that, The multimodal image acquisition module (8) includes an acquisition control unit (24) and multiple detection stations arranged sequentially along the production line conveyor belt (1). The detection stations include a transmitted light imaging unit (21), a reflected light imaging unit (22), and a polarization stress imaging unit (23). The transmitted light imaging unit (21) includes a uniform backlight source and a first industrial camera located on the opposite side of the building glass panel (3) for acquiring transmitted light images of the building glass panel (3) to detect bubbles and impurities. The reflected light imaging unit (22) includes a low-angle ring LED light source and a second industrial camera. The ring light source is tilted to illuminate the building glass panel (3) and is used to collect reflected light images to detect surface scratches, cracks and stains. The polarization stress imaging unit (23) includes a polarizer array and a third industrial camera. The polarizer array is set in front of the lens of the third industrial camera to acquire polarization stress images of the stress distribution and concentration state inside the building glass panel (3) based on different polarization angles. The acquisition control unit (24) is connected to the position sensor of the production line conveyor belt (1) and is used to control the camera, light source, parameters and viewing angle of the transmitted light imaging unit, reflected light imaging unit and polarized stress imaging unit to acquire multimodal image data in sequence when the building glass panel (3) arrives at the inspection station.

3. The building glass safety assessment system based on image recognition according to claim 2, characterized in that, The three-dimensional model reconstruction module (9) includes: The point cloud generation unit (12) is used to receive image data from multiple perspectives and calculate and generate high-density three-dimensional point cloud data representing the key structures of the surface and interior of the building glass panel (3) through feature point matching and triangulation algorithms. The surface meshing unit (13) is connected to the point cloud generation unit (12) and is used to convert the three-dimensional point cloud data into a continuous triangular mesh surface model through the Poisson reconstruction algorithm. The coordinate system construction and texture mapping unit (14) is connected to the surface meshing unit (13) to define a three-dimensional spatial coordinate system based on the physical characteristics of the building glass panel (3) in the triangular mesh surface model, and to map the original image texture to the model surface to form a measurable three-dimensional visual digital model with real texture.

4. The building glass safety assessment system based on image recognition according to claim 3, characterized in that, The defect identification and classification module (10) includes: The internal defect analysis unit (15) is connected to the transmitted light imaging unit (21) and is used to receive and process transmitted light images, identify bubbles and impurities through adaptive threshold segmentation and morphological analysis, and calculate their number, size, area ratio and spatial distribution density in the three-dimensional model. The surface defect extraction unit (16) is connected to the reflected light imaging unit (22) and is used to receive and process the reflected light image, extract potential linear defect regions using a directional edge detection algorithm, and obtain the sub-pixel precision contour coordinate sequence of the linear defect region. The feature quantization and classification unit (17) is connected to the surface defect extraction unit (16) and is used to fit the linear defect contour coordinate sequence into a feature line, and calculate the local width, depth, shape smoothness and width fluctuation amplitude along the feature line, and classify the linear defect into scratches or cracks based on the weighted decision model (18).

5. The building glass safety assessment system based on image recognition according to claim 4, characterized in that, The weighted decision model (18) is as follows: ； in, For normalized shape smoothness eigenvalues, For the normalized width eigenvalues, For normalized depth feature values, The normalized width fluctuation amplitude characteristic value; , , , These are the weight coefficients for the corresponding features, satisfying... ,and Configured as a negative value; The feature quantization and classification unit (17) presets a decision threshold. When the calculated A value is less than or equal to the decision threshold, the defect is determined to be a scratch. When the value is greater than the judgment threshold, the defect is determined to be a crack.

6. The building glass safety assessment system based on image recognition according to claim 5, characterized in that, The display panel (7) integrates a conflict detection module (11), which is connected to the multimodal image acquisition module (8), the defect identification and classification module (10) and the sorting execution module (25) respectively. It is used to verify the identification result of the defect identification and classification module (10) and send the final verification result to the sorting execution module (25).

7. The building glass safety assessment system based on image recognition according to claim 6, characterized in that, The collision detection module (11) is configured to perform the following steps: The defect identification and classification module (10) receives the identification result output by the defect identification and classification module (10), and extracts the three-dimensional surface geometry data and texture data of the corresponding coordinate region from the three-dimensional model reconstruction module (9) based on the multimodal image data corresponding to the identification result and the three-dimensional position; the multimodal image data includes at least the transmitted light image, reflected light image and polarized stress image of the same coordinate region; For identification results classified as cracks, verify whether there are local stress concentration patterns in the corresponding polarized stress image that match the mechanical characteristics of cracks; and for identification results classified as internal defects, verify whether the three-dimensional surface geometric data presents a continuous and flat shape. If so, the output of the defect identification and classification module (10) is determined to be correct; otherwise, the output of the defect identification and classification module (10) is determined to be incorrect. The multimodal image data corresponding to the erroneous results are processed, and the defects are classified again through the defect identification and classification module (10) after processing. Repeat the above operation until the defect identification and classification module (10) outputs the correct classification result.

8. The building glass safety assessment system based on image recognition according to claim 7, characterized in that, The verification steps for the local stress concentration pattern include: Based on the reflected light image, sub-pixel contours identified as crack defects are extracted in the region corresponding to the three-dimensional position; the major axis direction of the minimum bounding rectangle of the sub-pixel contour is defined as the reference direction of the crack. In the polarization stress image, an analysis region is defined with the three-dimensional position as the center; the stress gradient magnitude and direction of the pixels in the analysis region are calculated; pixels with gradient magnitudes exceeding a first predetermined threshold are identified, and it is determined whether a spatially continuous pixel cluster is formed. If a pixel cluster is formed, calculate the average gradient direction of the pixels within the pixel cluster and the angle between the average gradient direction and the reference orientation; if the angle is less than a second predetermined threshold, it is determined to be a local stress concentration pattern.

9. The building glass safety assessment system based on image recognition according to claim 8, characterized in that, The verification steps for whether the three-dimensional surface geometry data exhibits a continuous and flat shape include: Based on the three-dimensional surface geometry data, calculate the local surface curvature within a predetermined radius centered on the three-dimensional position; If the local surface curvature is less than the curvature threshold, the three-dimensional surface geometry is determined to be a continuous and flat shape. If the local surface curvature is greater than or equal to the curvature threshold, it is determined that the three-dimensional surface geometry does not present a continuous flat shape.

10. The building glass safety assessment system based on image recognition according to claim 9, characterized in that, The conflict detection module (11) also includes an error case database (19) and a data optimization model (20). The error case database (19) is used to store error type-effective handling method pairs that are recorded by the conflict detection module (11) and finally verified by the defect identification and classification module (10); Each error type-effective processing method pair includes different error types output by the defect identification and classification module (10) and the processing method of the original multimodal image data corresponding to different error types; The data optimization model (20) outputs effective processing methods based on error types; The construction process of the data optimization model (20) is as follows: A large number of error type-processing method pairs are obtained and labeled by human experts; after labeling, the error type-processing method pairs are input into the neural network unit for iterative training to obtain a data optimization model (20) based on the error type output for processing the original multimodal image data.

Images