Visual detection system and detection method for images in tin bath

Abstract

The invention discloses a visual detection system and detection method for images in a tin bath, and relates to the technical field of float glass production, the visual detection system comprises an image acquisition module, the image acquisition module is used for acquiring image data of the tin bath and converting the image data into electric signals for transmission; the image processing and analyzing module comprises an image processing unit and an image analyzing unit, and the image processing unit is used for receiving the real-time image data from the image collecting module, preprocessing the real-time image data and transmitting the preprocessed real-time image data to the image analyzing unit; effective features can be automatically extracted from a large amount of data, and the position and the angle state of the head of the edge roller and key parameters such as the thickness and the width of a glass tape can be quickly and accurately recognized. And with the key data, the data can be transmitted to a cold end through a whole-plant intercommunication network, so that further'intelligent manufacturing 'is realized, and one-key board change at the cold end is realized. Besides, an alarm can be quickly generated after the numerical deviation exceeding the allowable value is generated, so that maintenance personnel can be informed in time when the detected abnormal condition is detected, the response speed and the maintenance efficiency of the system are improved, and the production efficiency and the product quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of float glass production, and more specifically to a visual inspection system and method for inspecting images in a tin bath. Background Art

[0002] The tin bath is a key part of the float glass production line. The glass ribbon is formed in the tin bath. In the float glass production process, the tin bath is used to spread the glass liquid on the molten metal tin to form glass; the edge drawing machine is one of the special equipment on the float glass production line, which is used to control the thickness and width of the glass ribbon. The edge drawing machine operates in the tin bath. The thickness and width of the glass ribbon are controlled by adjusting the position of the edge drawing machine head. The performance parameters of the edge drawing machine, such as the rotation speed of the edge drawing wheel, the frame travel, the rotation angle, etc., need to be precisely controlled to meet the production needs of ultra-thin glass;

[0003] The existing tin bath image acquisition system is in the form of tin bath endoscope + tin bath external scope. The on-site camera collects the real-time image of the edge drawing machine head in the tin bath and transmits the image signal to the central control room. The operator in the central control room checks these video images at all times and judges whether the position of the edge drawing machine head meets the production process requirements based on experience.

[0004] However, in actual use, the tin bath structure is a wide front and narrow back trough shape, which makes it difficult to control the width and thickness of the glass ribbon. The tin bath runs at high temperature, and manual observation is not possible for a long time, so accidents such as plate swing, plate breakage, and full trough cannot be discovered in time. Affected by multiple factors such as environment, light, production process, and noise, the signal-to-noise ratio of the detection system is generally low. Due to the high viscosity and slow flow of glass liquid, there is a serious time lag. Machine vision surface defect detection, especially online detection, requires huge data volume, much redundant information, and high feature space dimension.

[0005] In addition, the existing system mainly relies on the operator to observe the position and angle of the edge drawing machine head through an endoscope and an external scope. The operator needs to judge whether the position of the edge drawing machine head meets the production process requirements based on experience. Due to the different experience and proficiency of the operators, it will affect the stability of production. The camera only transmits the picture to the TV wall in the control room to facilitate the operator's manual observation and recording;

[0006] However, during the production process, the molten glass continuously flows from the melting furnace into the tin bath and spreads and flattens on the tin bath surface to form a continuous glass ribbon. After passing through the tin bath, this continuous glass ribbon enters the annealing furnace for annealing. Since the glass ribbon is continuous, through image recognition, it is only possible to identify whether there are teeth marks on the surface of the glass ribbon and whether the position of the edge drawing machine is abnormal. However, the continuous glass ribbon produced still needs to be cut and processed later. Only identifying the teeth marks on the surface not only cannot identify the proportion of waste glass in the glass ribbon, but also cannot help the subsequent glass cutting process. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a tin bath image visual inspection system and inspection method.

[0008] The present invention provides a tin bath image visual detection system and detection method, comprising an image acquisition module, wherein the image acquisition module is used to collect image data of the tin bath and convert it into an electrical signal for transmission;

[0009] An image processing and analysis module, the image processing and analysis module comprises an image processing unit and an image analysis unit, the image processing unit is used to receive the real-time image data from the image acquisition module, perform preprocessing, and transmit the preprocessed real-time image data to the image analysis unit;

[0010] The image analysis unit is used to receive the real-time image data preprocessed by the image processing unit, and analyze the preprocessed real-time image data to obtain the tin bath detection result, the tin bath detection result specifically includes whether the position of the edge drawing machine head is abnormal and the proportion of waste glass. If the position of the edge drawing machine head is abnormal, an alarm signal is generated and transmitted;

[0011] The alarm module is used to receive the alarm signal of the image analysis and processing module and transmit it to the operation platform and mobile phone terminal of the maintenance personnel.

[0012] As an optional embodiment: the specific working steps of the image acquisition module are as follows:

[0013] Select an industrial camera, and adjust the parameters of the industrial camera using preset parameters;

[0014] Obtain parameter data of the tin bath, including the length, width and height data of the tin bath;

[0015] According to the parameter data of the tin bath, determine the installation quantity and installation position of the industrial camera, and install the industrial camera according to the determined installation quantity and installation position;

[0016] A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath.

[0017] The image data of all industrial cameras are converted into electrical signals and transmitted to the image processing and analysis module.

[0018] As an optional embodiment: the specific steps of determining the installation quantity and installation position of the industrial camera based on the parameter data of the tin bath are as follows:

[0019] Obtain the horizontal field of view angle N of the industrial camera;

[0020] According to the formula The horizontal coverage area Q of the industrial camera is calculated, where W is the distance from the industrial camera to the tin bath, which is the preset distance;

[0021] According to the formula The number of industrial cameras installed in the length direction of the tin bath is calculated and obtained as n1, where L is the length of the tin bath, and Δ is the overlapping part of each camera coverage area, which is 10% of the horizontal coverage area Q;

[0022] According to the formula and Calculate the number of industrial cameras installed in the direction of the widest part of the tin bath, n2, and the number of industrial cameras installed in the direction of the narrowest part of the tin bath, n3;

[0023] Where L2 is the width of the tin bath at its widest point, and L3 is the width of the tin bath at its narrowest point;

[0024] Along the length direction of the tin bath, starting from one end, the industrial cameras are installed with the horizontal coverage area Q of the industrial cameras cut out as intervals to ensure that the entire length direction is covered and the coverage areas of adjacent industrial cameras overlap by Δ, and the installation position of the industrial camera in the length direction of the tin bath is obtained. The method for obtaining the installation position of the industrial camera in the width direction of the tin bath is the same as the method for obtaining the installation position of the industrial camera in the length direction of the tin bath;

[0025] Add the number of industrial cameras n1 to n2 and n3 to get the total number of installed industrial cameras.

[0026] As an optional embodiment: the specific working steps of the image processing unit are as follows:

[0027] Receive the real-time image data of each industrial camera obtained from the image acquisition module, and convert all the real-time image data into a processable digital image format;

[0028] Select one of all industrial cameras as a reference camera;

[0029] The horizontal direction is set as the x-axis, and the vertical direction is set as the y-axis. The positive direction of the x-axis is consistent with the moving direction of the glass ribbon, and the positive direction of the y-axis can be consistent with the width direction of the tin bath.

[0030] According to the installation position of each industrial camera, their coordinates are calibrated in the coordinate system;

[0031] Detect feature points in the images acquired by each industrial camera; for example, use SIFT or ORB algorithms; extract feature points from images acquired by adjacent cameras;

[0032] Match the feature points of the images obtained by adjacent industrial cameras to find the corresponding feature point pairs; the FLANN matching method can be used;

[0033] For each pair of images obtained by adjacent industrial cameras, the RANSAC algorithm is used to calculate the antipodal matrix H according to the matched feature point pairs;

[0034] According to the calculated isographic matrix H, each image is perspective transformed, transformed into the coordinate system of the reference camera, and the transformed coordinates are normalized to the pixel coordinate system of the image to obtain the transformation result;

[0035] According to the transformation results of adjacent industrial camera images, their image overlapping areas are determined, and image fusion is performed on the overlapping areas;

[0036] For each pixel point in the overlapping area, its final value is calculated according to the fusion algorithm to obtain a complete fused real-time image of the tin bath.

[0037] As an optional embodiment: the specific steps of performing perspective transformation on each image, transforming it into the coordinate system of the reference camera, and normalizing the transformed coordinates into the pixel coordinate system of the image are:

[0038] According to the formula Complete the perspective transformation of each image, where (X, Y) is the coordinate point in the original image captured by each industrial camera, (X 1 , Y 1 ) is the coordinate point after transformation;

[0039] According to the formula and Calculate the coordinates (X 2 , Y 2 ).

[0040] As an optional embodiment: for each pixel point in the overlapping area, the specific steps of calculating its final value according to the fusion algorithm are as follows:

[0041] For each pixel in the overlapping area, the coordinates (X 2 , Y 2 ), according to the formula;

[0042] L1(X 2 , Y 2 )=0.8×L2(X 2 , Y 2 )+0.2×L3(X 2 , Y 2 ), calculated and obtained;

[0043] Where L2(X 2 , Y 2 ) is the pixel coordinate of the first adjacent image in the overlapping area, L3(X 2 , Y 2 ) is the pixel coordinate of the second adjacent image in the overlapping area.

[0044] As an optional embodiment: the specific working steps of the image analysis unit are:

[0045] The real-time image of the fused tin bath is obtained, and then the template features of the edge drawing machine head are obtained;

[0046] Extract the contour information of the edge drawing machine head and calculate its coordinate position in the real-time image of the tin bath;

[0047] Real-time acquisition of the coordinate position of the edge drawing machine head in the real-time image of the tin bath, including the horizontal position;

[0048] The position threshold range of the edge drawing machine head is pre-set, and the real-time horizontal position of the edge drawing machine head is compared with the position threshold. If the real-time horizontal position of the edge drawing machine head exceeds the position threshold range of the edge drawing machine head, an alarm signal is generated and transmitted to the alarm module.

[0049] As an optional embodiment: the specific working steps of the image analysis unit also include:

[0050] Binarization is performed on the real-time image of the tin bath to separate the tooth mark area from the background, and the pixel value of the tooth mark area is set to 1 and the background pixel value is set to 0, so as to highlight the tooth mark area;

[0051] Obtaining the boundary coordinates of the tooth mark area, and obtaining the area data of the tooth mark area according to the boundary coordinates;

[0052] For each tooth mark area, find its nearest glass strip edge point, according to the formula;

[0053] Calculate and obtain the distance d between the tooth mark area and the nearest edge point of the glass strip;

[0054] Where (X 0 , Y 0 ) is the coordinate of the center point of the tooth mark area, AX 0 +BY 0 +C is the straight line equation of the edge of the glass strip; the coordinates of the center point of the tooth mark area can be obtained by calculating the average position of all pixel points on the tooth mark contour;

[0055] The tooth mark area is regarded as a waste glass area, and the area of ​​the waste glass is calculated according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass strip. The area of ​​the waste glass is used as an alarm signal and transmitted to the alarm module.

[0056] The image processing and analysis module detects the position of the glass strip in the tin bath through the electrical signal of the image acquisition module, and transmits the position signal of the glass strip in the tin bath to the external cold end wire control system, so as to realize the integration of the tin bath edge drawing machine, the main transmission and the cold end wire control;

[0057] It should also be noted that, based on the extracted edge information and feature points, the pixel coordinates of the glass ribbon in the image are calculated and converted into actual object plane coordinates;

[0058] The calculated glass ribbon position signal is transmitted to the external cold end wire control system to realize the integrated control of the tin bath edge drawing machine, main drive and cold end wire control;

[0059] As an optional embodiment: the specific steps of calculating the area of ​​the waste glass according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass ribbon are as follows:

[0060] A threshold of tooth mark distance is set in advance. If the tooth mark distance d is less than the threshold of the tooth mark distance, it indicates that the tooth mark is close to the edge of the glass strip. Then the boundary of the tooth mark area is extended outward to the edge of the glass strip to obtain the abnormal area. If the tooth mark distance d is greater than the threshold of the tooth mark distance, the area of ​​the tooth mark area is taken as the abnormal area.

[0061] Obtain the center point coordinates of each tooth mark area, calculate the distance d1 between the center point coordinates of all adjacent tooth mark areas, and if the distance d1 is less than the threshold of the tooth mark distance, use the center point of the distance between the two adjacent tooth mark areas as the new diagonal endpoint, construct a square, use the distance d1 between the center point coordinates of the adjacent tooth mark areas as the length of the square, and use the new square as the abnormal area;

[0062] A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath.

[0063] Because the time interval collected by the image acquisition module is the time required for the glass ribbon to pass through a fixed length of the tin bath, the length S of the glass ribbon passing through the tin bath between the collection intervals of the image acquisition module is obtained, and the width of the glass ribbon is multiplied by the length S to obtain the total area K of the glass ribbon during this time interval. The abnormal area is divided by the total area K of the glass ribbon during this time interval to obtain the area ratio of waste glass in the glass ribbon during this time interval.

[0064] As an optional embodiment: the specific working steps of the alarm module are as follows:

[0065] Receive alarm signals in real time, analyze the received alarm signals, and extract the alarm type, alarm location, and alarm time;

[0066] Generate detailed alarm text information based on the parsed alarm information;

[0067] The generated alarm text information is transmitted to the maintenance personnel's operating platform mobile terminal through the wireless communication module.

[0068] Beneficial effects: It can automatically extract effective features from a large amount of data, quickly and accurately identify the position and angle status of the edge drawing machine head, as well as key parameters such as the thickness and width of the glass ribbon. The alarm module can promptly notify maintenance personnel when an abnormal situation is detected, thereby improving the system's response speed and maintenance efficiency, reducing production accidents caused by improper manual operation, reducing production costs, and improving production efficiency and product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of the management system of the present invention. DETAILED DESCRIPTION

[0070] Application scenario: In actual use, the existing system mainly relies on the operator to observe the position and angle of the edge drawing machine head through endoscopes and external scopes. The operator needs to judge whether the position of the edge drawing machine head meets the production process requirements based on experience. Due to the different experience and proficiency of operators, it will affect the stability of production. The camera only transmits the picture to the TV wall in the control room to facilitate the operator's manual observation and recording;

[0071] However, during the production process, the molten glass continuously flows from the melting furnace into the tin bath and spreads and flattens on the tin bath surface to form a continuous glass ribbon. After passing through the tin bath, this continuous glass ribbon enters the annealing furnace for annealing. Since the glass ribbon is continuous, through image recognition, it is only possible to identify whether there are teeth marks on the surface of the glass ribbon and whether the position of the edge drawing machine is abnormal. However, the continuous glass ribbon produced still needs to be cut and processed later. Only identifying the teeth marks on the surface not only cannot identify the proportion of waste glass in the glass ribbon, but also cannot help the subsequent glass cutting process.

[0072] like Figure 1 As shown: a tin bath image visual detection system and detection method, including an image acquisition module, the image acquisition module is used to collect image data of the tin bath and convert it into electrical signals for transmission;

[0073] An image processing and analysis module, the image processing and analysis module comprises an image processing unit and an image analysis unit, the image processing unit is used to receive the real-time image data from the image acquisition module, perform preprocessing, and transmit the preprocessed real-time image data to the image analysis unit;

[0074] The image analysis unit is used to receive the real-time image data pre-processed by the image processing unit, and analyze the pre-processed real-time image data to obtain the tin bath detection result, which specifically includes whether the position of the edge drawing machine head is abnormal and the proportion of waste glass. If the position of the edge drawing machine head is abnormal, an alarm signal is generated and transmitted; it should be noted that in the float glass production line, the edge drawing machine mainly controls the thickness and width of the glass ribbon. Specifically, the edge drawing machine pulls the glass ribbon floating on the tin liquid surface forward and controls the thickness and width of the glass ribbon through the rotation and position adjustment of its edge drawing wheel. In addition, the edge drawing machine also plays a role in stabilizing the operation of the glass ribbon;

[0075] The alarm module is used to receive the alarm signal of the image analysis and processing module and transmit it to the mobile phone terminal of the maintenance personnel.

[0076] High-resolution industrial cameras and infrared thermal imaging technology can work stably in high-temperature environments and accurately capture the shape and position information of the glass ribbon. Stereoscopic vision technology combines multi-view observation to provide three-dimensional data of the glass ribbon, allowing the system to more accurately control the width and thickness of the glass ribbon and reduce control errors caused by structural irregularities. These technologies improve the signal-to-noise ratio of the system and reduce the impact of environmental factors such as light and noise on the detection results, ensuring that high accuracy and stability can be maintained in complex environments, and accidents such as plate swing, plate breakage, and full slots can be discovered in a timely manner to ensure the continuity and safety of production.

[0077] It can automatically extract effective features from large amounts of data, quickly and accurately identify the position and angle status of the edge drawing machine head, as well as key parameters such as the thickness and width of the glass ribbon. The alarm module can promptly notify maintenance personnel when an abnormal situation is detected, thereby improving the system's response speed and maintenance efficiency, reducing production accidents caused by improper manual operation, reducing production costs, and improving production efficiency and product quality.

[0078] As an optional embodiment: the specific working steps of the image acquisition module are as follows:

[0079] Select an industrial camera, and use preset parameters to adjust the parameters of the industrial camera; it should be noted that the preset parameters specifically include resolution, frame rate, exposure time, and interface type;

[0080] Obtain parameter data of the tin bath, including the length, width and height data of the tin bath;

[0081] According to the parameter data of the tin bath, determine the installation quantity and installation position of the industrial camera, and install the industrial camera according to the determined installation quantity and installation position;

[0082] A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath.

[0083] The real-time image data of all industrial cameras are converted into electrical signals and transmitted to the image processing and analysis module.

[0084] It should also be noted that by reasonably determining the installation position and number of cameras, it can be ensured that all key areas in the tin bath are covered to avoid detection blind spots; multiple cameras can collect images from different angles, and combined with the three-dimensional reconstruction algorithm, the three-dimensional data of the glass ribbon can be obtained more accurately, thereby improving the detection accuracy of the width and thickness of the glass ribbon; the tin bath has an irregular structure and operates at high temperatures, and a reasonable camera layout can reduce the impact of environmental factors on the detection results and improve the stability and reliability of the system; by optimizing the number and position of cameras, the complexity and cost of the system can be reduced while meeting the detection needs.

[0085] As an optional embodiment: the specific steps of determining the installation quantity and installation position of the industrial camera based on the parameter data of the tin bath are as follows:

[0086] Get the horizontal field of view angle N of the industrial camera. It should be noted that it is usually determined by the camera lens and can be divided into horizontal field of view angle and vertical field of view angle. The larger the field of view angle, the wider the area covered by the camera. Due to the special nature of the tin bath, the vertical field of view angle does not need to be considered, so only the horizontal field of view angle of the camera needs to be considered.

[0087] According to the formula The horizontal coverage area Q of the industrial camera is calculated, where W is the distance from the industrial camera to the tin bath, which is the preset distance. It should be noted that the preset distance is the safe distance of the industrial camera because the temperature in the tin bath is relatively high. The preset distance is obtained according to the manufacturer's settings of the industrial camera.

[0088] According to the formula The number of industrial cameras installed in the length direction of the tin bath is calculated and obtained as n1, where L is the length of the tin bath, and Δ is the overlapping part of each camera coverage area, which is 10% of the horizontal coverage area Q;

[0089] According to the formula and Calculate the number of industrial cameras installed in the direction of the widest part of the tin bath, n2, and the number of industrial cameras installed in the direction of the narrowest part of the tin bath, n3;

[0090] Where L2 is the width of the tin bath at its widest point, and L3 is the width of the tin bath at its narrowest point;

[0091] Along the length direction of the tin bath, starting from one end, the industrial cameras are installed with the horizontal coverage area Q of the industrial cameras cut out as intervals to ensure that the entire length direction is covered and the coverage areas of adjacent industrial cameras overlap by Δ, and the installation position of the industrial camera in the length direction of the tin bath is obtained. The method for obtaining the installation position of the industrial camera in the width direction of the tin bath is the same as the method for obtaining the installation position of the industrial camera in the length direction of the tin bath;

[0092] Add the number of industrial cameras n1 to n2 and n3 to get the total number of installed industrial cameras.

[0093] It should be noted that the tin bath is a key part in the float glass production line, and the glass ribbon forming process inside it needs real-time monitoring and precise detection to ensure the quality of the glass and the stability of the production process. Determining the installation quantity and location of industrial cameras is the basis and prerequisite for achieving this goal. Through the reasonable layout of cameras, key parameters such as the thickness and width of the glass ribbon can be obtained in real time, providing accurate data support for subsequent control and adjustment, thereby ensuring the smooth progress of the production process. Reasonable camera layout can promptly detect accidents such as plate swing, broken plate, and full tank, ensuring that these abnormal situations are captured and identified in time, so as to take corresponding measures to avoid the occurrence and expansion of accidents and ensure production safety.

[0094] As an optional embodiment: the specific working steps of the image processing unit are as follows:

[0095] Receive the real-time image data of each industrial camera obtained from the image acquisition module, and convert all the real-time image data into a processable digital image format; it should be noted that the specific formats include BMP and JPEG;

[0096] Select one of all industrial cameras as a reference camera. It should be noted that a camera located at the center or one end of the tin bath is usually selected. The imaging plane of the reference camera is used as a reference to establish a plane rectangular coordinate system. The origin of the coordinate system can be set at the center of the reference camera.

[0097] The horizontal direction is set as the x-axis, and the vertical direction is set as the y-axis. The positive direction of the x-axis is consistent with the moving direction of the glass ribbon, and the positive direction of the y-axis can be consistent with the width direction of the tin bath.

[0098] According to the installation position of each industrial camera, their coordinates are calibrated in the coordinate system; it should be noted that, for example, if a camera is installed at one end of the tin bath, the horizontal distance from the reference camera is L, and the vertical distance is H, then the position of the camera in the coordinate system is (x=L, y=H);

[0099] Detect feature points in the images acquired by each industrial camera; for example, use SIFT or ORB algorithms; extract feature points from images acquired by adjacent cameras;

[0100] Match the feature points of the images obtained by adjacent industrial cameras to find the corresponding feature point pairs; the FLANN matching method can be used; it should be noted that by detecting and matching feature points in each image, the corresponding relationship between adjacent images can be accurately found, which is the basis for image stitching. Without accurate feature point matching, the stitched image will have problems such as misalignment and ghosting;

[0101] For each pair of images obtained by adjacent industrial cameras, the RANSAC algorithm is used to calculate the antipodal matrix H according to the matched feature point pairs;

[0102] According to the calculated isographic matrix H, each image is perspective transformed, transformed into the coordinate system of the reference camera, and the transformed coordinates are normalized to the pixel coordinate system of the image to obtain the transformation result;

[0103] According to the transformation results of adjacent industrial camera images, their image overlapping areas are determined, and image fusion is performed on the overlapping areas;

[0104] For each pixel point in the overlapping area, its final value is calculated according to the fusion algorithm to obtain a complete fused real-time image of the tin bath.

[0105] It should be noted that the image is perspective transformed according to the homography matrix, and the coordinates are normalized to the pixel coordinate system to ensure that the images captured by different cameras are aligned in the same coordinate system. This step is the prerequisite for achieving seamless stitching and can avoid obvious seams and discontinuities in the images at the stitching points. By stitching the images captured by multiple cameras, a complete panoramic image can be obtained, covering the entire inspection area of ​​the tin bath. This allows subsequent glass ribbon width and thickness inspections and edge drawing machine head position analysis to be performed in a unified view, avoiding blind spots and errors in inspections due to viewing angle limitations, and improving the comprehensiveness and accuracy of inspections.

[0106] As an optional embodiment: the specific steps of performing perspective transformation on each image, transforming it into the coordinate system of the reference camera, and normalizing the transformed coordinates into the pixel coordinate system of the image are:

[0107] According to the formula Complete the perspective transformation of each image, where (X, Y) is the coordinate point in the original image captured by each industrial camera, (X 1 , Y 1 ) is the coordinate point after transformation;

[0108] According to the formula and Calculate the coordinates (X 2 , Y 2 ).

[0109] As an optional embodiment: for each pixel point in the overlapping area, the specific steps of calculating its final value according to the fusion algorithm are as follows:

[0110] For each pixel in the overlapping area, the coordinates (X 2 , Y 2 ), according to the formula;

[0111] L1(X 2 , Y 2 )=0.8×L2(X 2 , Y 2 )+0.2×L3(X 2 , Y 2 ), calculated and obtained;

[0112] Where L2(X 2 , Y 2 ) is the pixel coordinate of the first adjacent image in the overlapping area, L3(X 2 , Y 2 ) is the pixel coordinate of the second adjacent image in the overlapping area.

[0113] As an optional embodiment: the specific working steps of the above-mentioned image analysis unit are:

[0114] The real-time image of the tin bath after fusion is obtained, and then the template features of the head of the edge drawing machine are obtained; it should be noted that the template features are obtained by using a high-resolution camera to take a clear image of the head when the edge drawing machine is working normally, and the template image is used as the template image to ensure that the template image can accurately reflect the features of the head, such as shape, texture, etc.;

[0115] Extract the contour information of the edge drawing machine head and calculate its coordinate position in the real-time image of the tin bath;

[0116] Real-time acquisition of the coordinate position of the edge drawing machine head in the real-time image of the tin bath, including the horizontal position;

[0117] The position threshold range of the edge drawing machine head is pre-set, and the real-time horizontal position of the edge drawing machine head is compared with the position threshold. If the real-time horizontal position of the edge drawing machine head exceeds the position threshold range of the edge drawing machine head, an alarm signal is generated and transmitted to the alarm module. It should be noted that the position threshold range is set according to production requirements and equipment parameters; it should be noted that the abnormal position of the edge drawing machine head may cause uneven force on the glass ribbon during the drawing process, thereby affecting the thickness uniformity of the glass. For example, if the head position deviates from the center line, it may cause one side of the glass ribbon to be thicker and the other side to be thinner, which cannot meet the quality requirements of the product. The head in an abnormal position may have abnormal contact or friction with the glass, resulting in scratches, embossing and other defects on the glass surface, reducing the appearance quality and market competitiveness of the product. Therefore, when the position of the edge drawing machine is abnormal, an alarm needs to be issued in time to prompt.

[0118] As an optional embodiment: the specific working steps of the image analysis unit also include:

[0119] The real-time image of the tin bath is binarized, the teeth mark area is separated from the background, the pixel value of the teeth mark area is set to 1, the background pixel value is set to 0, and the teeth mark area is highlighted; it should be noted that in the float glass production line, "teeth mark" usually refers to a defect, that is, the trace or imprint left by the mechanical contact of the glass during the production process. This trace may be caused by the uneven surface of the component or attachments (such as tin slag, broken glass, etc.) when the glass contacts certain components on the production line (such as rollers). This tooth mark will affect the appearance quality of the glass, so it is necessary to try to avoid or reduce its occurrence during the production process;

[0120] The boundary coordinates of the tooth mark area are obtained, and the area data of the tooth mark area is obtained according to the boundary coordinates; it should be noted that the area and perimeter of the tooth mark are calculated, and the area can be obtained by counting the number of pixels in the tooth mark area, and the perimeter can be obtained by calculating the length of the tooth mark contour;

[0121] For each tooth mark area, find its nearest glass strip edge point, according to the formula;

[0122] Calculate and obtain the distance d between the tooth mark area and the nearest edge point of the glass strip;

[0123] Where (X 0 , Y 0 ) is the coordinate of the center point of the tooth mark area, AX 0 +BY 0 +C is the straight line equation of the edge of the glass strip; the coordinates of the center point of the tooth mark area can be obtained by calculating the average position of all pixel points on the tooth mark contour; it should also be noted that A, B and C are coefficients obtained by least squares straight line fitting, and the specific calculation method is to first extract a number of points from the edge of the glass strip as sample points, obtain the mean x1 of the x coordinates of all sample points and the mean y1 of the y coordinates, calculate the covariance A and variance B of the x coordinates and y coordinates of all sample points, and C is obtained by calculation according to the formula C=-Ax1-By1;

[0124] The tooth mark area is regarded as a waste glass area, and the area of ​​the waste glass is calculated according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass strip. The area of ​​the waste glass is used as an alarm signal and transmitted to the alarm module.

[0125] As an optional embodiment: the specific steps of calculating the area of ​​the waste glass according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass ribbon are as follows:

[0126] A threshold of tooth mark distance is set in advance. If the tooth mark distance d is less than the threshold of the tooth mark distance, it indicates that the tooth mark is close to the edge of the glass strip. Then the boundary of the tooth mark area is extended outward to the edge of the glass strip to obtain the abnormal area. If the tooth mark distance d is greater than the threshold of the tooth mark distance, the area of ​​the tooth mark area is taken as the abnormal area.

[0127] Obtain the center point coordinates of each tooth mark area, calculate the distance d1 between the center point coordinates of all adjacent tooth mark areas, and if the distance d1 is less than the threshold of the tooth mark distance, use the center point of the distance between the two adjacent tooth mark areas as the new diagonal endpoint, construct a square, use the distance d1 between the center point coordinates of the adjacent tooth mark areas as the length of the square, and use the new square as the abnormal area; it should be noted that if the tooth mark areas are close to each other, the glass ribbon cannot be further cut between the tooth mark areas to obtain the product, so the abnormal area needs to be recalculated;

[0128] A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath.

[0129] Because the time interval collected by the image acquisition module is the time required for the glass ribbon to pass through a fixed length of the tin bath, the length S of the glass ribbon passing through the tin bath between the collection intervals of the image acquisition module is obtained, and the width of the glass ribbon is multiplied by the length S to obtain the total area K of the glass ribbon during this time interval. The abnormal area is divided by the total area K of the glass ribbon during this time interval to obtain the area ratio of waste glass in the glass ribbon during this time interval.

[0130] It should be noted that subsequent cutting suggestions can also be generated based on the area ratio of waste glass in the glass ribbon during this time interval; the area ratio of waste glass during this time interval is compared with historical data. If the area ratio of waste glass is significantly higher than the historical average, it means that the current area ratio of waste glass is high; if it is equal to or lower than the historical average, it can be considered that the area ratio of waste glass is normal or low.

[0131] In the case where the waste glass area accounts for a high proportion, it is recommended to cut the glass into small-sized products first, so that the waste glass area can be avoided more flexibly and the yield rate can be improved. Small-sized products have a relatively high tolerance for waste glass. Even if there is waste glass in some areas, it will not affect the performance of the entire product.

[0132] In the case where the waste glass area accounts for a low proportion, there are more high-quality areas in the glass ribbon, so the glass can be cut into large-size products first.

[0133] As an optional embodiment: the specific working steps of the alarm module are as follows:

[0134] Receive alarm signals in real time, analyze the received alarm signals, and extract the alarm type, alarm location, and alarm time;

[0135] For example, the edge drawing machine head position over-limit alarm signal includes the actual position coordinates of the head and the deviation value exceeding the threshold; the tooth mark distance over-limit alarm signal includes the specific position of the tooth mark and the actual distance from the edge of the glass ribbon, and the time point when the abnormality occurs;

[0136] Generate detailed alarm text information based on the analyzed alarm information; for example, the alarm text information of the edge drawing machine head position exceeding the limit is "The edge drawing machine head position exceeds the limit, the current horizontal position is 1200mm, which exceeds the threshold range (1000-1100mm), and the deviation value is 100mm; the alarm text information of the tooth mark distance exceeding the limit is that the tooth mark produced by the edge drawing machine on the glass strip is too close to the edge of the glass strip, the tooth mark position is (x=500mm, y=300mm), and the actual distance from the edge is 5mm, which is lower than the safety distance threshold of 10mm;

[0137] The generated alarm text information is transmitted to the mobile terminal of the maintenance personnel through the wireless communication module;

[0138] It should be noted that the image acquisition module is responsible for converting the real-time scene in the tin bath into electrical signal transmission; the image processing unit in the image processing and analysis module performs pre-processing such as denoising and enhancement on the acquired image data, and the image analysis unit obtains the tin bath detection results based on the processed data, such as the position of the edge drawing machine head, glass ribbon thickness and width and other parameters; the alarm module generates an alarm signal and notifies maintenance personnel when an abnormality is detected, thereby realizing full-process automated control from image acquisition to control adjustment and alarm, and improving production efficiency and product quality.

[0139] Working principle:

[0140] An image processing and analysis module, the image processing and analysis module comprises an image processing unit and an image analysis unit, the image processing unit is used to receive the real-time image data from the image acquisition module, perform preprocessing, and transmit the preprocessed real-time image data to the image analysis unit;

[0141] The image analysis unit is used to receive the real-time image data pre-processed by the image processing unit, and analyze the pre-processed real-time image data to obtain the tin bath detection result, which specifically includes whether the position of the edge drawing machine head is abnormal and the proportion of waste glass. If the position of the edge drawing machine head is abnormal, an alarm signal is generated and transmitted; it should be noted that in the float glass production line, the edge drawing machine mainly controls the thickness and width of the glass ribbon. Specifically, the edge drawing machine pulls the glass ribbon floating on the tin liquid surface forward and controls the thickness and width of the glass ribbon through the rotation and position adjustment of its edge drawing wheel. In addition, the edge drawing machine also plays a role in stabilizing the operation of the glass ribbon;

[0142] The alarm module is used to receive the alarm signal of the image analysis and processing module and transmit it to the mobile phone terminal of the maintenance personnel.

[0143] High-resolution industrial cameras and infrared thermal imaging technology can work stably in high-temperature environments and accurately capture the shape and position information of the glass ribbon. Stereoscopic vision technology combines multi-view observation to provide three-dimensional data of the glass ribbon, allowing the system to more accurately control the width and thickness of the glass ribbon and reduce control errors caused by structural irregularities. These technologies improve the signal-to-noise ratio of the system and reduce the impact of environmental factors such as light and noise on the detection results, ensuring that high accuracy and stability can be maintained in complex environments, and accidents such as plate swing, plate breakage, and full slots can be discovered in a timely manner to ensure the continuity and safety of production.

[0144] It can automatically extract effective features from large amounts of data, quickly and accurately identify the position and angle status of the edge drawing machine head, as well as key parameters such as the thickness and width of the glass ribbon. The alarm module can promptly notify maintenance personnel when an abnormal situation is detected, thereby improving the system's response speed and maintenance efficiency, reducing production accidents caused by improper manual operation, reducing production costs, and improving production efficiency and product quality.

[0145] The above are only preferred implementations of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical staff in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of this template.

Claims

1. A tin bath image visual inspection system, characterized in that: It includes an image acquisition module, which is used to collect image data of the tin bath and convert it into electrical signals for transmission; An image processing and analysis module, the image processing and analysis module comprises an image processing unit and an image analysis unit, the image processing unit is used to receive the real-time image data from the image acquisition module, perform preprocessing, and transmit the preprocessed real-time image data to the image analysis unit; The image analysis unit is used to receive the real-time image data preprocessed by the image processing unit, and analyze the preprocessed real-time image data to obtain the tin bath detection result, the tin bath detection result specifically includes whether the position of the edge drawing machine head is abnormal and the proportion of waste glass. If the position of the edge drawing machine head is abnormal, an alarm signal is generated and transmitted; The alarm module is used to receive the alarm signal of the image analysis and processing module and transmit it to the operation platform and mobile phone terminal of the maintenance personnel.

2. The tin bath image visual inspection system according to claim 1, characterized in that: The specific working steps of the image acquisition module are as follows: Select an industrial camera, and adjust the parameters of the industrial camera using preset parameters; Obtain parameter data of the tin bath, including the length, width and height data of the tin bath; According to the parameter data of the tin bath, determine the installation quantity and installation position of the industrial camera, and install the industrial camera according to the determined installation quantity and installation position; A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath. The real-time image data of all industrial cameras are converted into electrical signals and transmitted to the image processing and analysis module.

3. The tin bath image visual inspection system according to claim 2, characterized in that: The specific steps of determining the number and location of the industrial cameras to be installed based on the parameter data of the tin bath are as follows: Get the horizontal field of view angle N of the industrial camera; According to the formula The horizontal coverage area Q of the industrial camera is calculated, where W is the distance from the industrial camera to the tin bath, which is the preset distance; According to the formula The number of industrial cameras installed in the length direction of the tin bath is calculated and obtained as n1, where L is the length of the tin bath, and Δ is the overlapping part of each camera coverage area, which is 10% of the horizontal coverage area Q; According to the formula and Calculate the number of industrial cameras installed in the direction of the widest part of the tin bath, n2, and the number of industrial cameras installed in the direction of the narrowest part of the tin bath, n3; Where L2 is the width of the tin bath at its widest point, and L3 is the width of the tin bath at its narrowest point; Along the length direction of the tin bath, starting from one end, the industrial cameras are installed with the horizontal coverage area Q of the industrial cameras cut out as intervals to ensure that the entire length direction is covered and the coverage areas of adjacent industrial cameras overlap by Δ, and the installation position of the industrial camera in the length direction of the tin bath is obtained. The method for obtaining the installation position of the industrial camera in the width direction of the tin bath is the same as the method for obtaining the installation position of the industrial camera in the length direction of the tin bath; Add the number of industrial cameras n1 to n2 and n3 to get the total number of installed industrial cameras.

4. The tin bath image visual inspection system according to claim 2, characterized in that: The specific working steps of the image processing unit are as follows: Receive the real-time image data of each industrial camera obtained from the image acquisition module, and convert all the real-time image data into a processable digital image format; Select one of all industrial cameras as a reference camera; The horizontal direction is set as the x-axis, and the vertical direction is set as the y-axis. The positive direction of the x-axis is consistent with the moving direction of the glass ribbon, and the positive direction of the y-axis can be consistent with the width direction of the tin bath. According to the installation position of each industrial camera, their coordinates are calibrated in the coordinate system; Detect feature points in the images acquired by each industrial camera; for example, use SIFT or ORB algorithms; extract feature points from images acquired by adjacent cameras; Match the feature points of the images obtained by adjacent industrial cameras to find the corresponding feature point pairs; the FLANN matching method can be used; For each pair of images obtained by adjacent industrial cameras, the RANSAC algorithm is used to calculate the antipodal matrix H according to the matched feature point pairs; According to the calculated isographic matrix H, each image is perspective transformed, transformed into the coordinate system of the reference camera, and the transformed coordinates are normalized to the pixel coordinate system of the image to obtain the transformation result; According to the transformation results of adjacent industrial camera images, their image overlapping areas are determined, and image fusion is performed on the overlapping areas; For each pixel point in the overlapping area, its final value is calculated according to the fusion algorithm to obtain a complete fused real-time image of the tin bath.

5. The tin bath image visual inspection system according to claim 4, characterized in that: The specific steps of performing perspective transformation on each image, transforming it into the coordinate system of the reference camera, and normalizing the transformed coordinates into the pixel coordinate system of the image are as follows: According to the formula Complete the perspective transformation of each image, where (X, Y) is the coordinate point in the original image captured by each industrial camera, (X 1 , Y 1 ) is the coordinate point after transformation; According to the formula and Calculate the coordinate X in the normalized pixel coordinate system 2 , Y 2 ).

6. The tin bath image visual inspection system according to claim 4, characterized in that: The specific steps of calculating the final value of each pixel in the overlapping area according to the fusion algorithm are as follows: For each pixel in the overlapping area, the coordinates (X 2 , Y 2 ), according to the formula; L1(X 2 , Y 2 )=0.8×L2(X 2 , Y 2 )+0.2×L3(X 2 , Y 2 ), calculated and obtained; Where L2(X 2 , Y 2 ) is the pixel coordinate of the first adjacent image in the overlapping area, L3(X 2 , Y 2 The pixel coordinates of the second adjacent image in the overlapping area.

7. A tin bath image visual inspection system and inspection method according to claim 4, characterized in that: The specific working steps of the image analysis unit are: The real-time image of the fused tin bath is obtained, and then the template features of the edge drawing machine head are obtained; Extract the contour information of the edge drawing machine head and calculate its coordinate position in the real-time image of the tin bath; Real-time acquisition of the coordinate position of the edge drawing machine head in the real-time image of the tin bath, including the horizontal position; The position threshold range of the edge drawing machine head is pre-set, and the real-time horizontal position of the edge drawing machine head is compared with the position threshold. If the real-time horizontal position of the edge drawing machine head exceeds the position threshold range of the edge drawing machine head, an alarm signal is generated and transmitted to the alarm module.

8. The tin bath image visual inspection system according to claim 7, characterized in that: The specific working steps of the image analysis unit also include: Binarization is performed on the real-time image of the tin bath to separate the tooth mark area from the background, and the pixel value of the tooth mark area is set to 1 and the background pixel value is set to 0, so as to highlight the tooth mark area; Obtaining the boundary coordinates of the tooth mark area, and obtaining the area data of the tooth mark area according to the boundary coordinates; For each tooth mark area, find its nearest glass strip edge point, according to the formula; Calculate and obtain the distance d between the tooth mark area and the nearest edge point of the glass strip; Where (X0, Y0) is the coordinate of the center point of the tooth mark area, AX0+BY0+C is the straight line equation of the edge of the glass strip; the coordinate of the center point of the tooth mark area can be obtained by calculating the average position of all pixel points on the tooth mark contour; The tooth mark area is regarded as a waste glass area, and the area of ​​the waste glass is calculated according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass strip. The area of ​​the waste glass is used as an alarm signal and transmitted to the alarm module.

9. The method for visual inspection of images in a tin bath according to claim 7, characterized in that: The specific steps of calculating the area of ​​the waste glass according to the distance d, the area of ​​the tooth mark area and the total area of ​​the glass ribbon are as follows: A threshold of tooth mark distance is set in advance. If the tooth mark distance d is less than the threshold of the tooth mark distance, it indicates that the tooth mark is close to the edge of the glass strip. Then the boundary of the tooth mark area is extended outward to the edge of the glass strip to obtain the abnormal area. If the tooth mark distance d is greater than the threshold of the tooth mark distance, the area of ​​the tooth mark area is taken as the abnormal area. Obtain the center point coordinates of each tooth mark area, calculate the distance d1 between the center point coordinates of all adjacent tooth mark areas, and if the distance d1 is less than the threshold of the tooth mark distance, use the center point of the distance between the two adjacent tooth mark areas as the new diagonal endpoint, construct a square, use the distance d1 between the center point coordinates of the adjacent tooth mark areas as the length of the square, and use the new square as the abnormal area; A time interval is preset, which is the time required for the glass ribbon to pass through a fixed length of the tin bath. Every time this time interval passes, all industrial cameras capture image data in the tin bath. Because the time interval collected by the image acquisition module is the time required for the glass ribbon to pass through a fixed length of the tin bath, the length S of the glass ribbon passing through the tin bath between the collection intervals of the image acquisition module is obtained, and the width of the glass ribbon is multiplied by the length S to obtain the total area K of the glass ribbon during this time interval. The abnormal area is divided by the total area K of the glass ribbon during this time interval to obtain the area ratio of waste glass in the glass ribbon during this time interval.

10. The tin bath image visual inspection system according to claim 1, characterized in that: The specific working steps of the alarm module are as follows: Receive alarm signals in real time, analyze the received alarm signals, and extract the alarm type, alarm location, and alarm time; Generate detailed alarm text information based on the parsed alarm information; The generated alarm text information is transmitted to the maintenance personnel operation platform and mobile phone terminal through the wireless communication module.

Images