A method and system for visual inspection of steel billet surface based on 3D point cloud technology

Abstract

The present invention belongs to the field of computer vision technology and discloses a method and system for visual inspection of steel billet surfaces based on 3D point cloud technology. The method comprises: collecting image data of the end face and upper and lower surfaces of the steel billet; sequentially processing image data from different angles to identify defect information present in the steel billet; associating and archiving production information and defect information of the steel billet, and utilizing a distributed database for traceable storage and data management; triggering an alarm signal based on the real-time steel billet defect detection results; and displaying the steel billet transportation, inspection, and analysis process in real time. By introducing 3D point cloud technology, the present invention can accurately locate defects on the steel billet surface, ensuring that the exact location of each defect is identified and recorded. It can detect tiny defects that are difficult to detect with the naked eye, such as microcracks and small pits. It can also accurately classify various types of defects, such as cracks, holes, and scratches, and assess their impact on product quality, helping production managers make decisions.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a steel billet surface visual detection method and system based on 3D point cloud technology. Background Art

[0002] With the development of modern manufacturing and automation technology, the steel production industry has increasingly stringent requirements for improving production efficiency and ensuring product quality. The steel production process requires continuous casting, a process in which molten steel is directly cast from a high-temperature furnace into solid billets through a series of cooling and solidification steps. It is one of the most important casting methods in modern steelmaking. During the continuous casting process, various defects are easily formed on the billet surface due to the complex operating environment and large differences in heat conduction. To ensure billet quality during the continuous casting process and reduce production costs, surface defect detection technology is particularly important.

[0003] Common surface defects in steel billets include cracks, pits, and de-squareness. Surface cracks can develop on the billet surface due to overcooling or uneven cooling. The depth and width of the cracks directly affect the mechanical properties of the billet. Due to the uneven cooling process, pits of uneven thickness or unevenness may appear on the surface, affecting the billet's quality and subsequent processing. De-squareness (i.e., deviation, deformation, or irregularity in the billet's shape) is a common quality issue in steel production. In steel production, the billet's shape must maintain a certain standard size to ensure good controllability during subsequent processing.

[0004] Traditional surface defect detection for steel billets relies primarily on manual labor. While this method allows for clear and intuitive visualization of surface defects and is simple to operate, it relies on manual judgment, resulting in low efficiency and susceptible to operator experience. Prolonged manual operation can lead to fatigue and reduced detection accuracy. Magnetic particle testing uses the leakage magnetic field generated by a magnetic field on the billet surface to detect defects. If cracks or other defects are present on the billet surface, the leakage magnetic field will attract magnetic particles to the defect. However, magnetic particle testing cannot effectively identify deeper cracks or defects, and the billet must be cleaned after use, otherwise it will affect subsequent production processes.

[0005] In recent years, 3D point cloud technology, a visual perception technology based on depth information, has gained widespread attention for its application in steel billet surface defect detection due to its superiority in surface defect detection. By acquiring three-dimensional point cloud data of the steel billet surface through technologies such as laser scanning and structured light scanning, combined with advanced image processing algorithms, surface defects can be effectively identified and analyzed. Therefore, inspection systems based on 3D point cloud technology have shown great potential in steel billet surface defect detection and other applications.

[0006] 3D point cloud technology has many advantages in the detection of surface defects in steel billets, especially in terms of accuracy and comprehensiveness, and is suitable for detecting defects with complex shapes. However, there are also some problems: (1) Data processing is complex. 3D point cloud data is usually very large and contains a large amount of three-dimensional coordinate information. Processing this data requires high computing power and complex algorithms. The requirements for hardware and software are high. Post-processing, such as noise removal, missing point filling, and point cloud registration, increases the complexity and time cost of calculation. (2) The cost is high. The price of 3D point cloud acquisition equipment is high, and the initial investment requirements for enterprises are large. The professional requirements for equipment maintenance and data processing also make the overall cost relatively expensive. (3) The requirements for the environment are high. 3D point cloud scanning equipment is usually sensitive to ambient lighting conditions. When the lighting conditions are poor or there is too much reflection or gloss on the surface of the steel billet, the scanning accuracy and data quality may be affected. This may cause the point cloud data in some areas to be missing or incomplete, affecting the accuracy of defect detection.

[0007] Therefore, how to provide a steel billet surface visual inspection method and system based on 3D point cloud technology is a problem that needs to be solved urgently. Summary of the Invention

[0008] The embodiments of the present invention provide a method and system for visual inspection of steel billet surface based on 3D point cloud technology to solve the above-mentioned technical problems existing in the prior art.

[0009] To provide a basic understanding of some aspects of the disclosed embodiments, the following is a brief summary. This summary is not intended to be an extensive review, identify key or critical elements, or delineate the scope of these embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the detailed description that follows.

[0010] According to a first aspect of an embodiment of the present invention, a method for visually inspecting a steel billet surface based on 3D point cloud technology is provided.

[0011] In one embodiment, the steel billet surface visual inspection method based on 3D point cloud technology includes:

[0012] Collect image data of the end face and upper and lower surfaces of the billet;

[0013] Process image data from different angles in sequence to identify defects in the billet;

[0014] Associate and archive the production information and defect information of steel billets, and use the distributed database for traceable storage and data management;

[0015] Trigger an alarm signal based on the real-time billet defect detection results;

[0016] Real-time display of the billet transportation, testing and analysis process.

[0017] In one embodiment, sequentially processing image data at different angles to identify defect information of the steel billet includes:

[0018] Preprocess the billet surface image, generate a depth image through gradient interpolation, compare the depth difference of the original surface, and identify the defects on the billet surface;

[0019] Preprocess the billet end face image, segment the image background through semantic segmentation, calculate the physical dimensions of the billet, and determine whether there is any off-squareness.

[0020] The defect points and de-squareness detection results of the upper and lower surfaces and end faces of the steel billet are integrated as the defect information of the steel billet.

[0021] In one embodiment, generating a depth image by gradient interpolation and comparing the depth difference of the original curved surface to identify defects on the surface of the steel billet includes:

[0022] The Sobel operator is used to calculate the gradient of each pixel in the billet surface image and generate a gradient map;

[0023] Set the gradient threshold, select the pixels in the gradient map that are smaller than the gradient threshold, use the pixel values ​​of the four adjacent known pixels in the rectangular grid to interpolate, and perform linear interpolation on the gradient map to obtain the depth image;

[0024] The depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0025] In one embodiment, the interpolation calculation formula is:

[0026] ;

[0027] Where, I ( x , y ) represents the interpolation value of the target pixel; I ( x i , y j ) represents the pixel values ​​of four known image points; ( x i , y j ) represents the coordinates of the image point, where .

[0028] In one embodiment, dividing the image background by semantic segmentation, calculating the physical size of the billet, and determining whether there is out-of-squareness includes:

[0029] Semantic analysis is used to find the end face features in the billet end face image, and the background in the billet end face image is separated to obtain the billet end face rectangle.

[0030] Calculate the difference between the two diagonals of the billet end rectangle. If the difference is greater than the preset diagonal threshold, it is marked as out of square. If the difference is less than or equal to the preset diagonal threshold, it is marked as normal.

[0031] Calculate the distance between the two side lines of the billet end rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0032] In one embodiment, the associating and archiving the production information and defect information of the steel billet and utilizing a distributed database for traceable storage and data management includes:

[0033] Obtain the defect identification results of the steel billet, extract the defect type and defect coordinates, associate them with the steel billet number and archive them, and record the production information and defect information of the steel billet;

[0034] Distributed storage technology will be used to store the surface quality data of steel billets using multiple hard disk memories;

[0035] Set preset time intervals to transmit test data regularly, maintain data availability and fault tolerance through multi-node replication and load balancing, and use timestamp and metadata management to ensure the integrity and traceability of test data.

[0036] According to a second aspect of an embodiment of the present invention, a steel billet surface visual inspection system based on 3D point cloud technology is provided.

[0037] In one embodiment, the steel billet surface visual inspection system based on 3D point cloud technology includes:

[0038] Image acquisition module, used to collect image data of the end face and upper and lower surfaces of the billet;

[0039] Image processing module, used to process image data from different angles in sequence and identify defects in the billet;

[0040] The data storage and management module is used to associate and archive the production information and defect information of steel billets, and use the distributed database for traceable storage and data management;

[0041] An alarm module is used to trigger an alarm signal based on the real-time billet defect detection results;

[0042] The display module is used to display the billet transportation, inspection and analysis process in real time.

[0043] In one embodiment, the image processing module includes: a surface image processing module, an end face image processing module and a defect information output module, wherein:

[0044] The surface image processing module is used to pre-process the billet surface image, generate a depth image by gradient interpolation, compare the depth difference of the original surface, and identify the defect points on the billet surface;

[0045] The end face image processing module is used to pre-process the end face image of the steel billet, divide the image background by semantic segmentation, calculate the physical size of the steel billet, and determine whether there is any off-squareness;

[0046] The defect information output module is used to integrate the defect points and out-of-squareness detection results of the upper and lower surfaces and end surface of the steel billet as the defect information of the steel billet.

[0047] In one embodiment, generating a depth image by gradient interpolation and comparing the depth difference of the original curved surface to identify defects on the surface of the steel billet includes:

[0048] The Sobel operator is used to calculate the gradient of each pixel in the billet surface image and generate a gradient map;

[0049] Set the gradient threshold, select the pixels in the gradient map that are smaller than the gradient threshold, use the pixel values ​​of the four adjacent known pixels in the rectangular grid to interpolate, and perform linear interpolation on the gradient map to obtain the depth image;

[0050] The depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0051] In one embodiment, dividing the image background by semantic segmentation, calculating the physical size of the billet, and determining whether there is out-of-squareness includes:

[0052] Semantic analysis is used to find the end face features in the billet end face image, and the background in the billet end face image is separated to obtain the billet end face rectangle.

[0053] Calculate the difference between the two diagonals of the billet end rectangle. If the difference is greater than the preset diagonal threshold, it is marked as out of square. If the difference is less than or equal to the preset diagonal threshold, it is marked as normal.

[0054] Calculate the distance between the two side lines of the billet end rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0055] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0056] 1. By introducing 3D point cloud technology, the present invention can locate defects on the surface of steel billets with high precision, ensuring that the exact position of each defect is identified and recorded. It can detect tiny defects that are difficult to detect with the naked eye, such as microcracks and small pits; it can accurately classify various types of defects, such as cracks, holes, scratches, etc., and evaluate their impact on product quality, helping production managers make decisions on whether the steel billets need to be repaired or reprocessed.

[0057] 2. This invention automates processes that traditionally rely on manual inspection, reducing the need for manual operations and thus saving significant labor costs. By minimizing misjudgments and rework caused by manual inspection, potential losses during the production process are reduced. It seamlessly integrates with existing production lines, robots, and automated equipment to enable full-process monitoring and automated repair. Upon detecting a defect, it provides real-time feedback to the production line, adjusting production parameters (such as temperature and pressure) to prevent recurrence. It can also identify potential production process issues, providing data support and suggestions for process improvements.

[0058] 3. By introducing 3D point cloud technology, the traditional steel manufacturing industry can achieve a high degree of automation and intelligence, driving the manufacturing industry towards high-tech, high-value-added development. This will help enhance the international competitiveness of the entire industry and promote the optimization and upgrading of the economic structure. Through automated defect detection, steel producers can reduce the costs associated with returns or repairs of defective products, thereby improving production efficiency and profit margins. More efficient production models will drive productivity improvements across the entire economic system.

[0059] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0061] Figure 1 This is a flow chart showing a method for visually inspecting a steel billet surface based on 3D point cloud technology according to an exemplary embodiment;

[0062] Figure 2 This is a principle block diagram of a steel billet surface visual inspection system based on 3D point cloud technology according to an exemplary embodiment;

[0063] Figure 3It is a logical schematic diagram of a method for visually inspecting a steel billet surface based on 3D point cloud technology according to an exemplary embodiment. DETAILED DESCRIPTION

[0064] The following description and accompanying drawings sufficiently illustrate the specific embodiments herein to enable those skilled in the art to practice them. Portions and features of some embodiments may be included in or substituted for portions and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims, including all available equivalents thereof. Herein, the terms "first," "second," and the like are used solely to distinguish one element from another and do not require or imply any actual relationship or order between these elements. In practice, the first element can also be referred to as the second element, and vice versa. Furthermore, the terms "comprise," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a structure, device, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such structure, device, or apparatus. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the structure, device, or apparatus comprising the element. The various embodiments herein are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Similar or identical parts between the various embodiments can be referenced to each other.

[0065] The terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like used herein to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are intended only to facilitate the description of this document and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the present invention. In the description herein, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, they can be mechanical or electrical connections, or they can be internal connections between two elements, they can be directly connected, or they can be indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to the specific circumstances.

[0066] As used herein, unless otherwise specified, the term "plurality" means two or more.

[0067] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0068] In this article, the term "and / or" is used to describe the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0069] It should be understood that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0070] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software so that the processor can call and execute the operations corresponding to the above modules.

[0071] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0072] Figure 1 An embodiment of the present invention's method for visually inspecting a steel billet surface based on 3D point cloud technology is shown.

[0073] In this optional embodiment, the steel billet surface visual inspection method based on 3D point cloud technology includes:

[0074] Step S101, collecting image data of the end face and upper and lower surfaces of the steel billet;

[0075] Step S102: sequentially process image data from different angles to identify defect information of the steel billet;

[0076] Step S103: Associating the production information and defect information of the archived steel billet, and using a distributed database for traceable storage and data management;

[0077] Step S104: triggering an alarm signal according to the real-time billet defect detection result;

[0078] Step S105: Real-time display of the billet transportation, testing and analysis process.

[0079] In this optional embodiment, when processing image data from different angles in sequence to identify defect information of the steel billet, the surface image of the steel billet can be preprocessed, and a depth image can be generated through gradient interpolation. The depth difference of the original surface can be compared to identify defect points on the surface of the steel billet; the end face image of the steel billet can be preprocessed, the image background can be divided through semantic segmentation, and the physical size of the steel billet can be calculated to determine whether there is any de-squareness; the defect points and de-squareness detection results of the upper and lower surfaces and end face of the steel billet are integrated as the defect information of the steel billet.

[0080] In this optional embodiment, when generating a depth image by gradient interpolation and comparing the depth difference of the original surface to identify defect points on the billet surface, the Sobel operator can be used to calculate the gradient of each pixel point in the billet surface image to generate a gradient map; a gradient threshold is set, and pixel points in the gradient map that are less than the gradient threshold are selected, and the pixel values ​​of four adjacent known pixels in a rectangular grid are used to obtain interpolation, and the gradient map is linearly interpolated to obtain a depth image; the depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than the preset depth threshold, it is marked as a small defect; if the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0081] In this optional embodiment, the interpolation calculation formula is:

[0082] ;

[0083] Where, I ( x , y ) represents the interpolation value of the target pixel; I ( x i , y j ) represents the pixel values ​​of four known image points; ( x i , y j ) represents the coordinates of the image point, where .

[0084] In this optional embodiment, when dividing the image background through semantic segmentation and calculating the physical size of the billet to determine whether there is any de-squareness, semantic analysis can be used to find the end face features in the billet end face image, separate the background in the billet end face image to obtain the billet end face rectangle; calculate the difference between the two diagonals of the billet end face rectangle, if the diagonal difference is greater than the preset diagonal threshold, it is marked as de-squareness, if the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; calculate the distance between the two side lines of the billet end face rectangle, if the side line distance is greater than the preset side line threshold, it is marked as bulging, if the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0085] In this optional embodiment, when associating and archiving the production information and defect information of the steel billet and using a distributed database for traceable storage and data management, the defect identification results of the steel billet can be obtained, the defect type and defect coordinates can be extracted, and the steel billet number can be associated and archived to record the production information and defect information of the steel billet; distributed storage technology will be used to store the surface quality data of the steel billet using multiple hard disk memories; a preset time interval will be set to transmit the inspection data regularly, and data availability and fault tolerance will be maintained through multi-node replication and load balancing, and timestamp and metadata management will be used to achieve complete and traceable inspection data.

[0086] Figure 2 An embodiment of a steel billet surface visual inspection system based on 3D point cloud technology of the present invention is shown.

[0087] In this optional embodiment, the steel billet surface visual inspection system based on 3D point cloud technology includes:

[0088] Image acquisition module 201, used to collect image data of the end face and upper and lower surfaces of the steel billet;

[0089] The image processing module 202 is used to sequentially process image data from different angles to identify defect information in the steel billet;

[0090] The data storage and management module 203 is used to associate and archive the production information and defect information of the steel billet, and use the distributed database for traceable storage and data management;

[0091] An alarm module 204 is configured to trigger an alarm signal based on the real-time billet defect detection result;

[0092] The display module 205 is used to display the process of billet transportation, testing and analysis in real time.

[0093] In this optional embodiment, the image processing module 202 includes: a surface image processing module (not shown in the figure), an end face image processing module (not shown in the figure) and a defect information output module (not shown in the figure), wherein the surface image processing module is used to pre-process the surface image of the steel billet, generate a depth image through gradient interpolation, compare the depth difference of the original surface, and identify defect points on the surface of the steel billet; the end face image processing module is used to pre-process the end face image of the steel billet, divide the image background through semantic segmentation, calculate the physical size of the steel billet, and determine whether there is any de-squareness; the defect information output module is used to integrate the defect points and de-squareness detection results of the upper and lower surfaces and the end face of the steel billet as defect information of the steel billet.

[0094] In this optional embodiment, the depth image is generated by gradient interpolation, and the depth difference of the original surface is compared to identify the defect points on the surface of the steel billet, which includes: using the Sobel operator to calculate the gradient of each pixel point in the steel billet surface image to generate a gradient map; setting a gradient threshold, selecting pixel points in the gradient map that are less than the gradient threshold, interpolating using the pixel values ​​of four adjacent known pixels in a rectangular grid, and linearly interpolating the gradient map to obtain a depth image; setting the depth image as an ideal surface, calculating the depth difference between the ideal surface and the original surface where the steel billet surface image is located, if the depth difference is less than the preset depth threshold, it is marked as a small defect, and if the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0095] In this optional embodiment, the image background is divided by semantic segmentation, and the physical size of the billet is calculated to determine whether there is any de-squareness, including: using semantic analysis to find end face features in the billet end face image, separating the background in the billet end face image to obtain the billet end face rectangle; calculating the difference between the two diagonals of the billet end face rectangle, if the diagonal difference is greater than the preset diagonal threshold, it is marked as de-squareness, if the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; calculating the distance between the two side lines of the billet end face rectangle, if the side line distance is greater than the preset side line threshold, it is marked as bulging, if the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0096] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0097] Depend on Figure 2 As shown, the present invention is composed of: an image acquisition module, an image processing module, a data storage and management module, an alarm module, a display module, etc.

[0098] (1) Image acquisition module: For off-square detection, a 2D area array camera is used to capture the end face of the billet at the beat specified by the staff, and a 3D camera is used to scan the upper and lower surfaces of the billet. After a billet is scanned, the data is stored in the server for post-processing.

[0099] (2) Image processing module: Detection of pits, cracks, etc. on the upper and lower surfaces of the steel billet. First, preprocessing is performed to remove noise, fill holes, and normalize the data to ensure that the subsequent detection algorithm can work better. Feature extraction is performed on the steel billet surface. First, the image is corrected to eliminate the influence of mechanical jitter. Then the gradient of the depth image is calculated and the gradient of each point is obtained by the Sobel operator. , where the weight in the x direction is set to 0.8 and the weight in the y direction is set to 0.2. Set a mask to exclude iron oxide scale with smaller grayscale values ​​in the grayscale image. Set a threshold, select points in the gradient image that are smaller than the threshold, and finally perform linear interpolation on the gradient image: For a target pixel (x, y), the pixel values ​​of the four adjacent known pixels in the rectangular grid can be used to obtain the interpolated value:

[0100] ;

[0101] The interpolated depth map is used as the ideal surface, and the depth difference is compared with the original surface. Defects smaller than 1 mm are classified as small defects, and defects larger than 1 mm are classified as large defects.

[0102] For end face off-square detection, we first pre-process the image to remove noise, then use semantic segmentation to find the end face of the billet and separate it from the background, and determine whether it is off-square based on the physical size of the billet. The entire end face is rectangular, and the difference between the two diagonals is If the difference is greater than the specified threshold, it is out of square, and the edge distance If the distance is greater than the specified threshold, it is considered as bulging (also out of square).

[0103] (3) Data storage and management module: Based on the results fed back by the image processing module, i.e., defect category and defect coordinates, each billet number will be associated and archived, and information such as time, process, and workstation will be recorded. Throughout the entire production process, each billet number will be used as key identification information to link the inspection results and billet production information for real-time management. Distributed storage technology is used for storage, i.e., the surface defect data server allows the storage of surface quality data (including defect images) on an independent server, which usually includes a RAID (multi-hard disk) large-capacity storage system to ensure fast access to long-term data. The database software is based on SQL-Server and Parsytec's image file system (a special image file system for storing image data). The database management software controls the automatic transmission of data from the inspection system to the SQDS. It automatically transmits data from the inspection system to the SQDS at a certain preset time interval. Through multi-node replication and load balancing, high data availability and fault tolerance are ensured. In order to ensure data integrity and traceability, timestamp and metadata management are used.

[0104] (4) Alarm module: If defects are detected after the steel billet is inspected, an alarm signal will be immediately issued in the form of a screen prompt, sound and light alarm, etc. to alert the staff.

[0105] (5) Real-time display module: The transportation process, inspection process, and analysis process of the steel billet will be presented in real time to help staff understand the surface defect detection status.

[0106] like Figure 3 As shown, it is a logical display diagram of image acquisition, processing, detection and display of the present invention. The upper and lower surfaces and end face images of the steel billet can be captured by 3D and 2D cameras, and defect detection and automatic display and storage can be achieved through image processing technology.

[0107] In summary, due to the limitations of manual inspection, factors such as fatigue and lack of concentration can easily introduce errors, leading to missed defects. This invention, through the introduction of 3D point cloud technology, data processing, and analysis, avoids potential errors that can occur during manual operation, ensuring the consistency and reliability of inspection results. It also significantly improves inspection efficiency, reduces labor costs, human errors, and the risk of misjudgment. By incorporating automated algorithms (such as image processing and deep learning) for defect identification and classification, this invention enables real-time defect detection on the billet surface while the production line operates at high speed, significantly improving inspection efficiency and reducing labor costs.

[0108] This invention enhances the system's ability to detect complex shapes and diverse defects. It adapts to the diverse morphologies of steel billet surfaces and can identify subtle surface changes or defects, such as cracks, scratches, and scale. 3D point cloud technology has strong recognition capabilities regardless of the defect's morphology. This invention provides real-time defect detection results, generates a three-dimensional model, and visualizes it on a computer, quickly locating problem areas and helping operators more accurately determine the severity and location of defects, achieving real-time performance and data visualization.

[0109] The present invention is not limited to the structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for visual inspection of steel billet surface based on 3D point cloud technology, characterized in that: include: Collect image data of the end face and upper and lower surfaces of the billet; Process image data from different angles in sequence to identify defects in the billet; Associate and archive the production information and defect information of steel billets, and use the distributed database for traceable storage and data management; Trigger an alarm signal based on the real-time billet defect detection results; Real-time display of the billet transportation, testing and analysis process; The sequential processing of image data at different angles to identify defect information of the steel billet includes: Preprocess the billet surface image, generate a depth image through gradient interpolation, compare the depth difference of the original surface, and identify the defects on the billet surface; Preprocess the billet end face image, segment the image background through semantic segmentation, calculate the physical dimensions of the billet, and determine whether there is any off-squareness. Integrate the defect points and out-of-squareness detection results of the upper and lower surfaces and end faces of the billet as the defect information of the billet; The generation of a depth image by gradient interpolation and comparison of the depth difference of the original surface to identify defects on the surface of the steel billet include: The Sobel operator is used to calculate the gradient of each pixel in the billet surface image and generate a gradient map; Set the gradient threshold, select the pixels in the gradient map that are smaller than the gradient threshold, use the pixel values ​​of the four adjacent known pixels in the rectangular grid to interpolate, and perform linear interpolation on the gradient map to obtain the depth image; The depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

2. The method for visual inspection of steel billet surface based on 3D point cloud technology according to claim 1, characterized in that: The calculation formula of the interpolation is: ； Where, I ( x , y ) represents the interpolation value of the target pixel; I ( x i , y j ) represents the pixel values ​​of four known image points; ( x i , y j ) represents the coordinates of the image point, where .

3. The method for visual inspection of steel billet surface based on 3D point cloud technology according to claim 2, characterized in that: The method of dividing the image background by semantic segmentation, calculating the physical size of the billet, and determining whether there is any off-squareness includes: Semantic analysis is used to find the end face features in the billet end face image, and the background in the billet end face image is separated to obtain the billet end face rectangle. Calculate the difference between the two diagonals of the billet end rectangle. If the difference is greater than the preset diagonal threshold, it is marked as out of square. If the difference is less than or equal to the preset diagonal threshold, it is marked as normal. Calculate the distance between the two side lines of the billet end rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

4. The method for visual inspection of steel billet surface based on 3D point cloud technology according to claim 1, characterized in that: The associated archiving of the production information and defect information of the steel billet and the use of a distributed database for traceable storage and data management include: Obtain the defect identification results of the steel billet, extract the defect type and defect coordinates, associate them with the steel billet number and archive them, and record the production information and defect information of the steel billet; Adopt distributed storage technology and use multiple hard disk memories to store the surface quality data of steel billets; Set preset time intervals to transmit test data regularly, maintain data availability and fault tolerance through multi-node replication and load balancing, and use timestamp and metadata management to ensure the integrity and traceability of test data.

5. A steel billet surface visual inspection system based on 3D point cloud technology, characterized in that: include: Image acquisition module, used to collect image data of the end face and upper and lower surfaces of the billet; Image processing module, used to process image data from different angles in sequence and identify defects in the billet; The data storage and management module is used to associate and archive the production information and defect information of steel billets, and use the distributed database for traceable storage and data management; An alarm module is used to trigger an alarm signal based on the real-time billet defect detection results; Display module, used to display the billet transportation, testing and analysis process in real time; The image processing module includes: a surface image processing module, an end face image processing module and a defect information output module, wherein: The surface image processing module is used to pre-process the billet surface image, generate a depth image by gradient interpolation, compare the depth difference of the original surface, and identify the defect points on the billet surface; The end face image processing module is used to pre-process the end face image of the steel billet, divide the image background by semantic segmentation, calculate the physical size of the steel billet, and determine whether there is any off-squareness; The defect information output module is used to integrate the defect points and out-of-squareness detection results of the upper and lower surfaces and end faces of the steel billet as the defect information of the steel billet; The generation of a depth image by gradient interpolation and comparison of the depth difference of the original surface to identify defects on the surface of the steel billet include: The Sobel operator is used to calculate the gradient of each pixel in the billet surface image and generate a gradient map; Set the gradient threshold, select the pixels in the gradient map that are smaller than the gradient threshold, use the pixel values ​​of the four adjacent known pixels in the rectangular grid to interpolate, and perform linear interpolation on the gradient map to obtain the depth image; The depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

6. The steel billet surface visual inspection system based on 3D point cloud technology according to claim 5, characterized in that: The method of dividing the image background by semantic segmentation, calculating the physical size of the billet, and determining whether there is any off-squareness includes: Semantic analysis is used to find the end face features in the billet end face image, and the background in the billet end face image is separated to obtain the billet end face rectangle. Calculate the difference between the two diagonals of the billet end rectangle. If the difference is greater than the preset diagonal threshold, it is marked as out of square. If the difference is less than or equal to the preset diagonal threshold, it is marked as normal. Calculate the distance between the two side lines of the billet end rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

Images