Billet surface visual detection method and system based on 3D point cloud technology

Abstract

The invention belongs to the technical field of computer vision, and discloses a steel billet surface visual detection method and system based on a 3D point cloud technology, and the method comprises the steps: collecting the image data of the end face, the upper surface and the lower surface of a steel billet; sequentially processing the image data at different angles, and identifying defect information existing in the steel billet; associating production information and defect information of the archived steel billets, and performing traceable storage and data management by using a distributed database; triggering an alarm signal according to a real-time billet defect detection result; and steel billet transportation, detection and analysis processes are displayed in real time. By introducing the 3D point cloud technology, defects on the surface of the steel billet can be positioned with high precision, it is ensured that the accurate position of each defect is recognized and recorded, and tiny defects, such as micro cracks and tiny pits, which are difficult to perceive by naked eyes can be detected; various types of defects such as cracks, holes and scratches can be accurately classified, the influence of the defects on the product quality is evaluated, and production management personnel are helped to make decisions.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly relates to a visual inspection method and system for the surface of steel billets based on 3D point cloud technology. Background Art

[0002] With the development of modern manufacturing and automation technologies, the steel production industry has increasingly strict requirements for improving production efficiency and ensuring product quality. In the steel production process, continuous casting technology is required. The continuous casting process of steel is a production process in which molten steel is directly cast into solid steel billets from a high-temperature furnace through a series of cooling and solidification processes. It is one of the most important casting methods in modern steel smelting. During the continuous casting process of steel, due to the complex operating environment and large differences in heat conduction, various defects are likely to form on the surface of the steel billet. In order to ensure the quality of the steel billet during the continuous casting process of steel and reduce the production costs of enterprises, surface defect detection technology is particularly important.

[0003] Common surface defects of steel billets include cracks, pits, out-of-square, etc. Due to supercooling or uneven cooling on the surface of the steel billet, surface cracks may occur. The depth and width of the cracks directly affect the mechanical properties of the steel billet. Due to the unevenness of the cooling process, pits with uneven thickness or uneven surface may appear on the surface, affecting the quality of the steel billet and subsequent processing. The out-of-square phenomenon of the steel billet (i.e., deviation, deformation, or irregularity of the shape of the steel billet) is one of the common quality problems in steel production. In steel production, the shape of the steel billet must maintain certain standard dimensions to ensure good controllability in subsequent processing.

[0004] Traditional surface defect detection of steel billets mainly relies on manual operation. The defects on the surface of the steel billet can be seen intuitively and clearly, and the operation is simple. However, it depends on manual judgment, has low efficiency and is easily affected by the experience of the operators. Long-term manual operation is prone to fatigue, reducing the accuracy of detection. The magnetic particle inspection method uses the leakage magnetic field generated on the surface of the steel billet by the magnetic field to detect defects. If there are cracks or other defects on the surface of the steel billet, the leakage magnetic field will adsorb the magnetic particles at the defects. For deeper cracks or defects, magnetic particle inspection cannot effectively identify them. After using magnetic particles, the steel billet needs to be cleaned, otherwise it will affect the subsequent production process.

[0005] In recent years, as a visual perception technology based on depth information, 3D point cloud technology has gradually received wide attention in the application of surface defect detection of steel billets due to its superiority in surface defect detection and other aspects. By using technologies such as laser scanning and structured light scanning to obtain the three-dimensional point cloud data of the surface of the steel billet and combining advanced image processing algorithms, surface defects can be effectively identified and analyzed. Therefore, the detection system based on 3D point cloud technology shows great potential in surface defect detection of steel billets and other aspects.

[0006] The 3D point cloud technology has many advantages in the detection of surface defects of steel billets, especially in terms of accuracy and comprehensiveness, and is suitable for detecting defects with complex shapes. However, there are also some problems: (1) The data processing is complex. 3D point cloud data is usually very large, containing a large amount of three-dimensional coordinate information. Processing this data requires high computing power and complex algorithms. It has high requirements for hardware and software. Post-processing operations such as noise removal, missing point filling, and point cloud registration increase the computational complexity and time cost. (2) The cost is relatively high. The price of 3D point cloud acquisition equipment is relatively high, requiring a large initial investment from enterprises. The professional requirements for equipment maintenance and data processing also make the overall cost relatively expensive. (3) It has high requirements for the environment. 3D point cloud scanning equipment is usually sensitive to environmental lighting conditions. When the lighting conditions are poor or there is too much reflection or gloss on the surface of the steel billet, it may affect the scanning accuracy and data quality. This may lead to missing or incomplete point cloud data in some areas, affecting the accuracy of defect detection.

[0007] Therefore, how to provide a visual inspection method and system for the surface of steel billets based on 3D point cloud technology is an urgent problem to be solved at present. Summary of the Invention

[0008] Embodiments of the present invention provide a visual inspection method and system for the surface of steel billets based on 3D point cloud technology to solve the above technical problems existing in the prior art.

[0009] To have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary part is not a general review, nor is it intended to identify key / important constituent elements or delineate the protection scope of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0010] According to the first aspect of the embodiments of the present invention, a visual inspection method for the surface of steel billets based on 3D point cloud technology is provided.

[0011] In one embodiment, the visual inspection method for the surface of steel billets based on 3D point cloud technology includes: Collect image data of the end face and upper and lower surfaces of the steel billet; Process the image data at different angles in sequence to identify the defect information existing in the steel billet; Associate and file the production information and defect information of the steel billet, and use a distributed database for traceable storage and data management; Trigger an alarm signal according to the real-time steel billet defect detection result; Real-time display the transportation, detection, and analysis process of the steel billet.

[0012] In one embodiment, the processing of the image data at different angles in sequence to identify the defect information existing in the steel billet includes: Preprocess the surface image of the steel billet, generate a depth image through gradient interpolation, compare the depth interpolation of the original surface, and identify the defect points existing on the surface of the steel billet; Preprocess the end-face image of the steel billet, divide the image background through semantic segmentation, calculate the physical size of the steel billet, and judge whether there is square deviation; Integrate the defect points and the square deviation detection results of the upper and lower surfaces and the end face of the steel billet as the defect information of the steel billet.

[0013] In one embodiment, the steps of generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying the defect points existing on the surface of the steel billet include: Calculate the gradient of each pixel point in the surface image of the steel billet using the Sobel operator to generate a gradient map; Set a gradient threshold, select the pixel points in the gradient map that are less than the gradient threshold, obtain the interpolation using the pixel values of the four adjacent known pixel points within the rectangular grid, and perform linear interpolation on the gradient map to obtain a depth image; Set the depth image as the ideal surface, calculate the depth difference between the ideal surface and the original surface where the surface image of the steel billet is located. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0014] In one embodiment, the calculation formula for the interpolation is: ; In the formula, I ( x , y ) represents the interpolation of the target pixel point; I ( x i , y j ) represents the pixel values of the four known image points; ( x i , y j ) represents the coordinates of the image points, where .

[0015] In one embodiment, the steps of dividing the image background through semantic segmentation, calculating the physical size of the steel billet, and judging whether there is square deviation include: Use semantic analysis to find the end-face features in the end-face image of the steel billet, and separate the background in the end-face image of the steel billet to obtain a rectangular end-face of the steel billet; Calculate the difference between the two diagonals of the rectangular end-face of the steel billet. If the diagonal difference is greater than the preset diagonal threshold, it is marked as having square deviation. If the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; Calculate the distance between two side lines of the rectangle at the end face of the billet. If the distance between the side lines is greater than the preset side line threshold, it is marked as bulging. If the distance between the side lines is less than or equal to the preset side line threshold, it is marked as normal.

[0016] In one embodiment, the production information and defect information of the associated archived billets, and the traceable storage and data management using a distributed database include: Obtain the defect recognition result of the billet, extract the defect type and defect coordinates, associate and archive them with the billet number, and record the production information and defect information of the billet; Adopt the distributed storage technology and use a multi-hard disk memory to store the surface quality data of the billet; Set a preset time interval, regularly transmit the detection data, maintain data availability and fault tolerance through multi-node replication and load balancing, and adopt timestamp and metadata management to achieve the integrity and traceability of the detection data.

[0017] According to the second aspect of the embodiments of the present invention, a billet surface vision detection system based on 3D point cloud technology is provided.

[0018] In one embodiment, the billet surface vision detection system based on 3D point cloud technology includes: An image acquisition module for acquiring image data of the end face and the upper and lower surfaces of the billet; An image processing module for sequentially processing image data at different angles to identify defect information existing in the billet; A data storage and management module for associating and archiving the production information and defect information of the billet, and performing traceable storage and data management using a distributed database; An alarm module for triggering an alarm signal according to the real-time billet defect detection result; A display module for real-time displaying the billet transportation, detection, and analysis processes.

[0019] In one embodiment, the image processing module includes: a surface image processing module, an end face image processing module, and a defect information output module, where The surface image processing module is used to preprocess the billet surface image, generate a depth image through gradient interpolation, compare the depth interpolation of the original surface, and identify defect points existing on the billet surface; The end face image processing module is used to preprocess the billet end face image, divide the image background through semantic segmentation, calculate the physical size of the billet, and determine whether there is square deviation; The defect information output module is used to integrate the defect points and square deviation detection results of the upper and lower surfaces and the end face of the billet as the defect information of the billet.

[0020] In one embodiment, generating a depth image through gradient interpolation and identifying defect points existing on the surface of a billet by comparing the depth interpolation of the original surface includes: Calculating the gradients of each pixel point in the surface image of the billet using a Sobel operator to generate a gradient map; Setting a gradient threshold, selecting the pixel points in the gradient map that are less than the gradient threshold, obtaining an interpolation using the pixel values of four adjacent known pixel points within a rectangular grid, and performing linear interpolation on the gradient map to obtain a depth image; Setting the depth image as an ideal surface, calculating the depth difference between the ideal surface and the original surface where the billet surface image is located. If the depth difference is less than a preset depth threshold, it is marked as a small defect; if the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0021] In one embodiment, dividing the image background through semantic segmentation, calculating the physical dimensions of the billet, and determining whether there is a square-off includes: Using semantic analysis to find the end face features in the end face image of the billet and separating the background in the end face image of the billet to obtain a rectangular shape of the billet end face; Calculating the difference between the two diagonals of the rectangular shape of the billet end face. If the diagonal difference is greater than a preset diagonal threshold, it is marked as square-off; if the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; Calculating the distance between the two side lines of the rectangular shape of the billet end face. If the side line distance is greater than a preset side line threshold, it is marked as bulging; if the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0022] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: 1. By introducing 3D point cloud technology, the present invention can accurately locate the defects on the surface of the billet, ensure that the exact positions of each defect are identified and recorded, be able to detect tiny defects that are difficult to detect by the naked eye, such as microcracks, small pits, etc.; be able to accurately classify various types of defects, such as cracks, holes, scratches, etc., and evaluate their impacts on product quality, helping production management personnel make decisions on whether to repair or reprocess the billet.

[0023] 2. The present invention automates the traditional process that relies on manual inspection, reduces the need for manual operations, thus saving a large amount of labor costs. By reducing misjudgments and rework caused by manual inspection, it reduces the possible losses during the production process; can be seamlessly docked with existing production lines, robots, and automation equipment to achieve full-process monitoring and automated repair; after detecting a defect, it can be real-time feedback to the production line to adjust production parameters (such as temperature, pressure, etc.) to avoid the recurrence of defects, and be able to identify potential production process problems, thereby providing data support and suggestions for process improvement.

[0024] 3. By introducing 3D point cloud technology, traditional steel manufacturing can achieve a high degree of automation and intelligence, promoting the development of the manufacturing industry towards high technology and high added value. This helps to enhance the international competitiveness of the entire industry and promote the optimization and upgrading of the economic structure. Through automated defect detection, steel producers can reduce the costs associated with defective product returns or repairs, thereby increasing production efficiency and profit margins. A more efficient production model will drive the productivity improvement of the entire economic system.

[0025] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present invention and, together with the specification, are used to explain the principles of the present invention.

[0027] Figure 1 is a flowchart of a billet surface visual inspection method based on 3D point cloud technology shown according to an exemplary embodiment; Figure 2 is a schematic block diagram of the principle of a billet surface visual inspection system based on 3D point cloud technology shown according to an exemplary embodiment; Figure 3 is a logical schematic diagram of a billet surface visual inspection method based on 3D point cloud technology shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following description and the accompanying drawings fully disclose specific embodiments herein, enabling those skilled in the art to practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents of the claims. In this document, terms such as "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a structure, device or equipment comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such structure, device or equipment. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the structure, device or equipment comprising the said element. The various embodiments herein are described in a progressive manner, with each embodiment highlighting the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other.

[0029] In this document, terms such as "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. This is only for the convenience of describing this document and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In the description herein, unless otherwise specified and defined, the terms "mounted", "connected", "coupled" shall be understood in a broad sense. For example, it may be a mechanical connection or an electrical connection, or may be the communication inside two elements. It may be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0030] In this document, unless otherwise stated, the term "plurality" means two or more.

[0031] In this document, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0032] In this document, the term "and / or" is a description of the associated relationship of an object, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B these three relationships.

[0033] It should be understood that although the steps in the flowchart are shown sequentially in the direction of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0034] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0035] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0036] Figure 1 An embodiment of a billet surface visual inspection method based on 3D point cloud technology of the present invention is shown.

[0037] In this alternative embodiment, the billet surface visual inspection method based on 3D point cloud technology includes: Step S101, collecting image data of the end face and the upper and lower surfaces of the billet; Step S102, sequentially processing the image data at different angles to identify the defect information existing in the billet; Step S103, associating and filing the production information and defect information of the billet, and using a distributed database for traceable storage and data management; Step S104, triggering an alarm signal according to the real-time billet defect detection result; Step S105, displaying the billet transportation, detection, and analysis process in real time.

[0038] In this alternative embodiment, when sequentially processing the image data at different angles to identify the defect information existing in the billet, the billet surface image can be preprocessed, a depth image is generated by gradient interpolation, the depth interpolation of the original surface is compared, and the defect points existing on the billet surface are identified; the billet end face image is preprocessed, the image background is divided by semantic segmentation, and the physical size of the billet is calculated to judge whether there is square-off; the defect points and the square-off detection results of the upper and lower surfaces and the end face of the billet are integrated as the defect information of the billet.

[0039] In this alternative embodiment, when generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying defect points on the billet surface, the gradient of each pixel point in the billet surface image can be calculated using the Sobel operator to generate a gradient map; a gradient threshold is set, pixel points in the gradient map that are less than the gradient threshold are selected, interpolation is obtained using the pixel values of four adjacent known pixel points within a rectangular grid, and linear interpolation is performed on the gradient map to obtain a depth image; the depth image is set as an ideal surface, and the depth difference between the ideal surface and the original surface where the billet surface image is located is calculated. If the depth difference is less than a preset depth threshold, it is marked as a small defect, and if the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0040] In this alternative embodiment, the calculation formula for the interpolation is as follows: ; In the formula, I ( x , y ) represents the interpolation of the target pixel point; I ( x i , y j ) represents the pixel values of four known image points; ( x i , y j ) represents the coordinates of the image points, where .

[0041] In this alternative embodiment, when segmenting the image background through semantic segmentation, calculating the physical dimensions of the billet, and determining whether there is squareness, semantic analysis can be used to find end-face features in the billet end-face image, and the background in the billet end-face image is separated to obtain a billet end-face rectangle; the difference between the two diagonals of the billet end-face rectangle is calculated. If the diagonal difference is greater than a preset diagonal threshold, it is marked as squareness, and if the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; the distance between the two side lines of the billet end-face rectangle is calculated. If the side-line distance is greater than a preset side-line threshold, it is marked as bulging, and if the side-line distance is less than or equal to the preset side-line threshold, it is marked as normal.

[0042] In this alternative embodiment, when associating and archiving the production information and defect information of the billet, and using a distributed database for traceable storage and data management, the defect identification result of the billet can be obtained, the defect type and defect coordinates are extracted, associated and archived with the billet number, and the production information and defect information of the billet are recorded; the distributed storage technology is used, and the surface quality data of the billet is stored using a multi-hard disk memory; a preset time interval is set, and the detection data is transmitted regularly. Through multi-node replication and load balancing, the data availability and fault tolerance are maintained, and timestamp and metadata management are used to achieve the integrity and traceability of the detection data.

[0043] Figure 2 An embodiment of a visual inspection system for the surface of a steel billet based on 3D point cloud technology of the present invention is shown.

[0044] In this alternative embodiment, the visual inspection system for the surface of a steel billet based on 3D point cloud technology includes: An image acquisition module 201 for acquiring image data of the end face and the upper and lower surfaces of the steel billet; An image processing module 202 for sequentially processing image data at different angles to identify defect information existing in the steel billet; A data storage and management module 203 for associating and filing the production information and defect information of the steel billet, and performing traceable storage and data management using a distributed database; An alarm module 204 for triggering an alarm signal according to the real-time steel billet defect detection result; A display module 205 for real-time displaying the process of steel billet transportation, detection and analysis.

[0045] In this alternative embodiment, the image processing module 202 includes: a surface image processing module (not shown in the figure), an end face image processing module (not shown in the figure), and a defect information output module (not shown in the figure). Among them, the surface image processing module is used for preprocessing the steel billet surface image, generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying defect points existing on the steel billet surface; the end face image processing module is used for preprocessing the steel billet end face image, dividing the image background through semantic segmentation, calculating the physical size of the steel billet, and judging whether there is square deviation; the defect information output module is used for integrating the defect points and the square deviation detection results of the upper and lower surfaces and the end face of the steel billet as the defect information of the steel billet.

[0046] In this alternative embodiment, the generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying defect points existing on the steel billet surface includes: calculating the gradient of each pixel point in the steel billet surface image using the Sobel operator to generate a gradient map; setting a gradient threshold, selecting the pixel points in the gradient map that are less than the gradient threshold, obtaining interpolation using the pixel values of four adjacent known pixel points within a rectangular grid, and performing linear interpolation on the gradient map to obtain a depth image; setting the depth image as an ideal surface, calculating the depth difference between the ideal surface and the original surface where the steel billet surface image is located. If the depth difference is less than a preset depth threshold, it is marked as a small defect, and if the depth difference is greater than the preset depth threshold, it is marked as a large defect.

[0047] In this alternative embodiment, the steps of dividing the image background through semantic segmentation, calculating the physical dimensions of the billet, and determining whether there is a square-off include: using semantic analysis to find the end face features in the billet end face image, separating the background in the billet end face image to obtain a billet end face rectangle; calculating the difference between the two diagonals of the billet end face rectangle. If the diagonal difference is greater than the preset diagonal threshold, it is marked as square-off. If the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; calculating the distance between the two side lines of the billet end face rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] As Figure 2 shown, the present invention consists of: an image acquisition module, an image processing module, a data storage and management module, an alarm module, a display module, etc.

[0050] (1) Image acquisition module: For square-off detection, a 2D area array camera is used to capture the billet end face at the rhythm specified by the staff, and a 3D camera is used to scan the upper and lower surfaces of the billet. After scanning a billet, the data is stored in the server for subsequent processing.

[0051] (2) Image processing module: For the detection of pits, cracks, etc. on the upper and lower surfaces of the billet. First, preprocessing is carried out, the purpose of which is to remove noise, fill holes, and standardize the data to ensure that the subsequent detection algorithms can work better. Feature extraction is performed on the billet surface. First, the picture is corrected to exclude the influence of mechanical jitter. Then, the gradient of the depth image is calculated, and the gradient of each point is obtained through the Sobel operator, where the weight in the x direction is set to 0.8 and the weight in the y direction is set to 0.2. Set the mask to exclude the scale oxide with a small gray value in the grayscale image. Set the threshold, select the points within the range less than the threshold in the gradient image, and finally perform linear interpolation on the gradient image: For a target pixel point (x,y), the interpolation can be obtained using the pixel values of the four adjacent known pixel points within the rectangular grid: ; The interpolated depth image is used as the ideal surface, and then the depth difference is compared with the original surface. Defects less than 1 mm are classified as small defects, and defects greater than 1 mm are classified as large defects.

[0052] For end face square-off detection, first, preprocessing is carried out to remove image noise, and then semantic segmentation is used to find the billet end face and separate it from the background, and it is determined whether there is a square-off according to the physical dimensions of the billet. The entire end face is a rectangle, and the difference between the two diagonals If the difference is greater than the specified threshold, it is out of square, and the edge distance If the distance is greater than the specified threshold, it is considered as bulging (also considered as off-square).

[0053] (3) Data storage and management module: According to the feedback results of the image processing module, i.e., defect category and defect coordinates, each billet number will be associated and archived, and the time, process, workstation and other information will be recorded. In the entire production process, each billet number will be used as key identification information to link the inspection results and billet production information for real-time management. Distributed storage technology is used for storage, i.e., the surface defect data server allows the storage of surface quality data (including defect images) on an autonomous server, which usually includes a RAID (multi-hard disk) large-capacity storage system to ensure fast access to long-term data. The database software is based on SQL-Server and Parsytec's image file system (a special image file system for storing image data). The database management software controls the automatic transmission of data from the inspection system to the SQDS. It automatically transmits data from the inspection system to the SQDS at a certain preset time interval. High availability and fault tolerance of data are ensured through multi-node replication and load balancing. In order to ensure the integrity and traceability of data, timestamp and metadata management are adopted.

[0054] (4) Alarm module: If defects are detected after the steel billet is inspected, an alarm signal will be immediately issued in the form of a screen prompt, sound and light alarm, etc. to alert the staff.

[0055] (5) Real-time display module: The transportation process, inspection process, and analysis process of the steel billet will be presented in real time to help staff understand the status of surface defect detection.

[0056] like Figure 3 As shown, it is a logical display diagram of image acquisition, processing, detection and display of the present invention. The upper and lower surfaces and end surface images of the steel billet can be obtained by capturing with 3D and 2D cameras, and defect detection and automatic display and storage can be achieved through image processing technology.

[0057] In summary, due to the limitations of manual inspection, factors such as personnel fatigue and concentration can easily introduce errors, resulting in missed defects. The present invention avoids possible mistakes in manual operation by introducing 3D point cloud technology, data processing and analysis, ensuring the consistency and reliability of the inspection results; at the same time, it significantly improves the inspection efficiency and reduces labor costs, human errors and misjudgment risks. The present invention combines automated algorithms (such as image processing, deep learning, etc.) for defect identification and classification, and can perform real-time defect detection on the billet surface while the production line is running at high speed, significantly improving inspection efficiency and reducing labor costs.

[0058] The present invention can enhance the system's detection ability for complex shapes and diverse defects, can adapt to the diverse morphologies on the surface of steel billets, and can identify in detail the subtle changes or defects on the surface, such as cracks, scratches, scale, etc. Regardless of the morphology of the defects, the 3D point cloud technology has strong recognition ability. The present invention can provide real-time defect detection results, generate a three-dimensional model and perform visual display on a computer, quickly locate the problem area, help the operator more accurately judge the severity and location of the defects, and achieve real-time performance and data visualization.

[0059] The present invention is not limited to the structures already described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A visual inspection method for the surface of steel billets based on 3D point cloud technology, characterized in that, It includes: Collecting image data of the end face and the upper and lower surfaces of the billet; Processing the image data at different angles in sequence to identify the defect information existing in the billet; Associating and archiving the production information and defect information of the billet, and using a distributed database for traceable storage and data management; Triggering an alarm signal according to the real-time billet defect detection result; Displaying the billet transportation, detection, and analysis processes in real time.

2. The visual inspection method for the surface of a steel billet based on 3D point cloud technology according to claim 1, wherein The processing of the image data at different angles in sequence to identify the defect information existing in the billet includes: Preprocessing the billet surface image, generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying the defect points existing on the billet surface; Preprocessing the billet end face image, dividing the image background through semantic segmentation, calculating the physical size of the billet, and judging whether there is square deviation; Integrating the defect points and the square deviation detection results of the upper and lower surfaces and the end face of the billet as the defect information of the billet.

3. The visual inspection method for the surface of a steel billet based on 3D point cloud technology according to claim 2, wherein, The generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying the defect points existing on the billet surface includes: Calculating the gradient of each pixel point in the billet surface image using the Sobel operator to generate a gradient map; Setting a gradient threshold, selecting the pixel points in the gradient map that are less than the gradient threshold, obtaining the interpolation using the pixel values of the four adjacent known pixel points within the rectangular grid, and performing linear interpolation on the gradient map to obtain a depth image; Setting the depth image as an ideal surface, calculating the depth difference between the ideal surface and the original surface where the billet surface image is located. If the depth difference is less than the preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

4. The visual inspection method for the surface of steel billets based on 3D point cloud technology according to claim 3, characterized in that, The calculation formula for the interpolation is: ； In the formula, I ( x , y ) represents the interpolation of the target pixel point; I ( x i , y j ) represents the pixel values of four known image points; ( x i , y j ) represents the coordinates of an image point, where .

5. The visual inspection method for the surface of a steel billet based on 3D point cloud technology according to claim 2, characterized in that, The dividing the image background through semantic segmentation, calculating the physical size of the billet, and judging whether there is square deviation includes: Using semantic analysis to find the end face features in the billet end face image, separating the background in the billet end face image to obtain the billet end face rectangle; Calculating the difference between the two diagonals of the billet end face rectangle. If the diagonal difference is greater than the preset diagonal threshold, it is marked as square deviation. If the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; Calculating the distance between the two side lines of the billet end face rectangle. If the side line distance is greater than the preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

6. The visual inspection method for the surface of steel billets based on 3D point cloud technology according to claim 1, characterized in that The associating and archiving the production information and defect information of the billet, and using a distributed database for traceable storage and data management includes: Obtaining the billet defect recognition result, extracting the defect type and defect coordinates, associating and archiving them with the billet number, and recording the production information and defect information of the billet; Adopting a distributed storage technology and using a multi-hard disk storage device to store the surface quality data of the billet; Setting a preset time interval, regularly transmitting the detection data, maintaining data availability and fault tolerance through multi-node replication and load balancing, and using time stamps and metadata management to achieve the integrity and traceability of the detection data.

7. A visual inspection system for the surface of steel billets based on 3D point cloud technology, characterized in that, It includes: An image acquisition module for collecting image data of the end face and the upper and lower surfaces of the billet; An image processing module for processing the image data at different angles in sequence to identify the defect information existing in the billet; A data storage and management module, which is used to associate and file the production information and defect information of billets, and perform traceable storage and data management using a distributed database; An alarm module, which is used to trigger an alarm signal according to the real-time billet defect detection results; A display module, which is used to display the billet transportation, detection and analysis processes in real time.

8. The visual inspection system for the surface of steel billets based on 3D point cloud technology according to claim 7, characterized in that, The image processing module includes: a surface image processing module, an end face image processing module and a defect information output module, where, The surface image processing module is used to preprocess the billet surface image, generate a depth image through gradient interpolation, compare the depth interpolation of the original surface, and identify the defect points existing on the billet surface; The end face image processing module is used to preprocess the billet end face image, divide the image background through semantic segmentation, calculate the physical size of the billet, and judge whether there is square deviation; The defect information output module is used to integrate the defect points and the square deviation detection results of the upper and lower surfaces and the end face of the billet as the defect information of the billet.

9. The visual inspection system for the surface of a steel billet based on 3D point cloud technology according to claim 8, wherein, The generating a depth image through gradient interpolation, comparing the depth interpolation of the original surface, and identifying the defect points existing on the billet surface includes: Calculating the gradients of each pixel point in the billet surface image using the Sobel operator to generate a gradient map; Setting a gradient threshold, selecting the pixel points in the gradient map that are less than the gradient threshold, obtaining the interpolation using the pixel values of four adjacent known pixel points in a rectangular grid, and performing linear interpolation on the gradient map to obtain a depth image; Setting the depth image as an ideal surface, calculating the depth difference between the ideal surface and the original surface where the billet surface image is located. If the depth difference is less than a preset depth threshold, it is marked as a small defect. If the depth difference is greater than the preset depth threshold, it is marked as a large defect.

10. The visual inspection system for the surface of steel billets based on 3D point cloud technology according to claim 8, characterized in that, The dividing the image background through semantic segmentation, calculating the physical size of the billet, and judging whether there is square deviation includes: Using semantic analysis to find the end face features in the billet end face image, and separating the background in the billet end face image to obtain a billet end face rectangle; Calculating the difference between the two diagonals of the billet end face rectangle. If the diagonal difference is greater than a preset diagonal threshold, it is marked as square deviation. If the diagonal difference is less than or equal to the preset diagonal threshold, it is marked as normal; Calculating the distance between the two side lines of the billet end face rectangle. If the side line distance is greater than a preset side line threshold, it is marked as bulging. If the side line distance is less than or equal to the preset side line threshold, it is marked as normal.

Images