Real-time online detection method and device for steel strip splitting based on image and storage medium

Abstract

The invention discloses an image-based real-time online detection method and device for steel belt strips and a storage medium. The method comprises the following steps: shooting the steel belt strips output by a steel belt slitting machine at regular time to obtain an image of a to-be-detected steel belt; according to the to-be-detected steel belt image and a defect detection model constructed by the system, whether the steel belt strips in the to-be-detected steel belt image have defects or not is judged, and when the steel belt strips in the to-be-detected steel belt image have the defects, the steel belt strip image of each steel belt strip with the defects is judged; matching each steel belt strip image obtained by segmentation with a defect identification model constructed in the system so as to identify the defect type of each steel belt strip and the position of each defect area in the steel belt strip image; and according to the position of each defect area in the corresponding steel strip splitting image, outline drawing is carried out on each defect area to obtain a defect graph, and the defect graph is displayed to a worker through a man-machine interaction display screen. According to the invention, real-time on-line detection of steel strip splitting can be realized.

Description

Technical Field

[0001] This invention relates to the field of online monitoring of production equipment, and more particularly to an image-based real-time online monitoring system and method for steel strip slitting. Background Technology

[0002] In defect detection on steel strip slitting machines, the process typically involves placing the slitting strips on a worktable after slitting, capturing images of each strip individually with a camera, and then using image processing technology to perform image detection. Defective strips are then marked based on the detection results. This method is unsuitable for online detection on steel strip slitting machines. Furthermore, given the large number and length of the steel strips, it's difficult to capture all sections of each strip in a single photograph, potentially leading to missed defects and inaccurate detection results. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, one of the objectives of this invention is to provide an image-based real-time online detection method for steel strip slitting, which can solve the problems of existing steel strip slitting methods being unable to perform online detection and having inaccurate detection results.

[0004] The second objective of this invention is to provide an image-based real-time online inspection device for steel strip slitting, which can solve the problems of existing steel strip slitting methods being unable to perform online inspection and having inaccurate inspection results.

[0005] The third objective of this invention is to provide a computer-readable storage medium that can solve the problems of existing steel strip slitting methods being unable to perform online detection and having inaccurate detection results.

[0006] One of the objectives of this invention is achieved through the following technical solution:

[0007] A real-time online detection method for steel strip slitting based on images, the method comprising:

[0008] Image acquisition steps: Periodically photograph the steel strip slits output by the steel strip slitting machine to obtain images of the steel strip to be tested; the images of the steel strip to be tested include multiple steel strip slits output by the steel strip slitting machine;

[0009] Defect detection steps: Based on the image of the steel strip to be tested and the defect detection model constructed by the system, it is determined whether there are defects in the steel strip slits in the image of the steel strip to be tested. When there are defects in the steel strip slits in the image of the steel strip to be tested, the steel strip slits associated with each defect area are obtained according to the position of the defect area in the image of the steel strip to be tested. Then, the image of the steel strip to be tested is segmented according to the steel strip slits with defects to obtain the steel strip slit image of each steel strip slit with defects.

[0010] Defect identification steps: Match each segmented steel strip image with the defect identification model built in the system to identify the defect type of each steel strip and the location of each defect region in the steel strip image;

[0011] Display steps: Based on the position of each defect area in the corresponding steel strip slitting image, the outline of each defect area is drawn to obtain a defect illustration. Then, each defect illustration is combined with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image and displayed to the staff through a human-computer interaction display screen.

[0012] Furthermore, the display step of marking each defect illustration in the corresponding steel strip slicing image specifically includes: marking each defect illustration in the corresponding steel strip slicing image according to different colors based on the defect type in each defect illustration; and calculating the size data of each defect area based on the outline of each defect.

[0013] Furthermore, the display step also includes: statistically analyzing the historical defect detection results of each steel strip and deduplicating and merging multiple defect areas of each steel strip to obtain the defect statistics of each steel strip; then matching the statistical results of each steel strip with preset defect level rules to obtain the defect level; and generating a corresponding audible and visual alarm notification based on the defect level.

[0014] Furthermore, the construction process of the defect detection model specifically includes: preprocessing each historical steel strip image that does not contain defects by statistically analyzing historical images of the steel strip to be tested in the system; extracting feature vectors from each historical steel strip image, then constructing a model based on a CNN detection network model and training the model based on the historical steel strip images to obtain the defect detection model; and dynamically updating the defect detection model constructed in the system based on newly acquired images of the steel strip to be tested.

[0015] Furthermore, the construction process of the defect type identification model specifically includes: acquiring multiple steel strip images of each defect type and preprocessing each steel strip image; dividing each steel strip image into grids to obtain multiple grids, and extracting features from each grid of each steel strip image based on the features of each defect type of the steel strip to obtain the feature vector of each steel strip image; then training the feature vector of each steel strip image based on a convolutional neural network model to construct a set of feature vectors of multiple steel strip images of each defect type; and constructing a defect identification model for different defect types based on the combination of the feature vectors of multiple steel strip images of each defect type.

[0016] Furthermore, the image data acquisition also includes: setting up a multimodal fusion lighting system at the process inspection point to provide different illuminations to supplement the camera, thereby acquiring steel strip images under different illumination modes; and extracting feature vectors from the steel strip images under different illumination modes to obtain feature maps of the steel strip images under different illumination modes, and fusing the feature maps of the steel strip images under different illumination modes to obtain the steel strip image to be tested; wherein, the illumination modes include: bright field illumination, dark field illumination, and coaxial light illumination.

[0017] Furthermore, the image data acquisition step also includes: preprocessing the acquired steel strip images under different lighting modes; wherein, the preprocessing includes flattening correction, data augmentation, and normalization processing; wherein, the flattening correction refers to unifying the shadows in the background of the steel strip image caused by uneven lighting or lens vignetting to produce a background with uniform brightness; the data augmentation refers to removing noise from the corresponding steel strip image by simulating the jitter, deviation, lighting fluctuation, and camera noise of the steel strip; the normalization processing refers to normalizing each steel strip image according to a preset image format and size.

[0018] Furthermore, it also includes an early warning step: when the cumulative time of defects in the corresponding steel strip slitting exceeds a preset time, a defect early warning notification is generated based on the defect type, cumulative duration of defects, defect start time, and defect end time of the corresponding steel strip slitting and fed back to the PLC industrial control computer of the steel strip slitting machine, so that the PLC industrial control computer of the steel strip slitting machine can predict the cause of defect formation based on the defect early warning notification and the expert system built based on the AI ​​model;

[0019] Simultaneously, based on the defect start time and defect end time of each steel strip slitting, the system acquires data signals collected by various sensor modules during the operation of the steel strip slitting machine. The system analyzes the changing trends of the monitoring data collected by each sensor to determine if there are any abnormalities in the corresponding equipment. Then, based on the cause of the defect and the abnormal state of the equipment, the system dynamically adjusts the corresponding equipment. Furthermore, based on the cause of the defect, the system dynamically fine-tunes the steel strip slitting process according to the predicted analysis. When the number of fine-tuning operations exceeds a preset number and the corresponding steel strip slitting defect still exists, an audible and visual alarm is issued or the machine is stopped. The sensor modules include tension sensors, vibration sensors, and acoustic emission sensors, which are used to acquire the tension of each frame of the steel strip slitting machine, the vibration spectrum of the cutter shaft and bearings, and the acoustic emission signals of cutter wear, respectively.

[0020] The second objective of this invention is achieved by the following technical solution:

[0021] The image-based real-time online detection device for steel strip slitting includes a memory and a processor. The memory stores a point cloud stitching program that runs on the processor. The point cloud stitching program is a computer program. When the processor executes the point cloud stitching program, it implements the steps of a point cloud stitching method for house layouts, as one of the objectives of this invention.

[0022] The third objective of this invention is achieved by the following technical solution:

[0023] A computer-readable storage medium storing a real-time online detection program for steel strip slitting, the real-time online detection program for steel strip slitting being a computer program, which, when executed by a processor, implements the steps of an image-based real-time online detection method for steel strip slitting as one of the objectives of this invention.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention takes a unified picture of multiple steel strips output by a steel strip slitting machine to form an overall image of the steel strip. First, a large-scale defect detection is performed on the steel strip image to determine whether there are defects in the steel strip slitting. Then, for steel strips with defects, the defect type, defect size, and contour positioning are further identified, thereby completing the real-time online detection of steel strip slitting. Attached Figure Description

[0026] Figure 1 The flowchart of the image-based real-time online detection method for steel strip slitting provided by the present invention is shown. Detailed Implementation

[0027] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0028] Example 1

[0029] This invention achieves defect detection of steel strip output from a steel strip slitting machine by combining images and models. At the same time, it links human-computer interaction to display visualized detection results to workers and combines an audible and visual alarm to provide audible and visual alarms for defects of different degrees, so as to realize real-time online detection of steel strip slitting.

[0030] Specifically, such as Figure 1 As shown, this invention provides an image-based real-time online detection method for steel strip slitting, comprising:

[0031] Step S1: Periodically photograph the steel strip slits output by the steel strip slitting machine to obtain images of the steel strip to be tested; the images of the steel strip to be tested include multiple steel strip slits output by the steel strip slitting machine.

[0032] The camera's data acquisition frequency can be determined by the encoder, matching the production speed of the strip slitting machine. Specifically, after the camera acquires the current strip image, the timing of the camera's next data acquisition is controlled by combining the encoder and the production speed of the strip slitting machine. This ensures that each acquired strip image represents a new strip slitting process, avoiding repeated acquisitions of the same strip and preventing duplicate subsequent inspection results.

[0033] Furthermore, steel strip slitting machines typically cut a wide strip of steel into multiple strips, which are then output simultaneously. Therefore, this invention utilizes a camera installed at the output point of the steel strip slitting machine to capture images of the multiple strips being tested, with the strips arranged side-by-side within the image. Moreover, when the output width of the steel strip slitting machine is wide, a single camera may not be able to cover all the strips. In this case, multiple cameras can be used, and image data can be simultaneously acquired by parallel cameras. These multiple images are then stitched together to obtain the final steel strip image.

[0034] Meanwhile, the image of the steel strip to be tested may include not only multiple steel strips but also background or other equipment images. Therefore, after acquiring the image of the steel strip to be tested, redundant equipment images or backgrounds are pre-removed after the steel strip is segmented and identified to ensure the removal of redundant features. Specifically, for example, a minimum rectangular frame is set according to the width of the segmented steel strips to delete redundant background features outside the rectangular frame. More preferably, the camera used to acquire the image of the steel strip to be tested can be an industrial camera to improve the resolution of the acquired image, such as a line scan camera or an area scan camera.

[0035] Step S2: Based on the image of the steel strip to be tested and the defect detection model constructed by the system, determine whether there are defects in the steel strip slits in the image of the steel strip to be tested. When there are defects in the steel strip slits in the image of the steel strip to be tested, determine the steel strip slits associated with each defect area based on the position of the defect area in the image of the steel strip to be tested. Then, based on the steel strip slits with defects, segment the image of the steel strip to be tested to obtain the steel strip slit image of each steel strip slit with defects.

[0036] Specifically, this invention acquires images and combines them with a model to detect defects in steel strip images, thereby determining whether defects exist in the steel strip segments within the images. Specifically, this invention is based on a CNN detection network model. It involves statistically analyzing historical steel strip images without defects in the system and preprocessing each historical image; extracting feature vectors from each historical image; then combining these with a CNN-based detection network model to construct a model, which is trained on the historical images to generate a defect detection model; and simultaneously, dynamically updating the defect detection model based on newly acquired steel strip images.

[0037] This invention does not require detecting the type and size of defects during defect detection. Instead, it identifies defective strips by detecting the entire steel strip image. These defective strips are then segmented from the image, and further defect type and size data are obtained from them. This step-by-step processing improves system efficiency and avoids performance degradation due to excessive data volume. Specifically, the CNN detection network model, such as YOLOv8, can be used for image defect detection.

[0038] Step S3: Match each segmented steel strip image with the defect recognition model built in the system to identify the defect type of each steel strip and the location of each defect region in the steel strip image.

[0039] Specifically, by segmenting the defective steel strip image from the steel strip image to be tested, the defective steel strip image and the defect recognition model are then used to identify the defect type. At the same time, the identified defect area is combined to delineate the defect outline and calculate the defect size.

[0040] Specifically, multiple images of steel strip slitting for each defect type are acquired and preprocessed. Each image is then divided into multiple grids, and feature vectors are extracted from each grid based on the features of each defect type. A convolutional neural network (CNN) model is then used to train the feature vectors of each image to construct a set of feature vectors for each defect type. Finally, a defect recognition model for different defect types is constructed by combining the feature vectors from multiple images for each defect type. Specifically, the CNN model, such as Vision Transformer or ResNeSt, can be used as the backbone network classification model to identify defect types in the steel strip slitting images. Defect types can be classified based on practical experience and may include scratches, pits, protrusions, curled edges, burrs, etc.

[0041] In addition to training the model mentioned above, during the defect type identification process, a 3D laser device can be simultaneously used to scan the steel strip image captured by the camera to obtain 3D point cloud models of multiple steel strip segments in the image. When a defect is determined in a particular steel strip segment, further auxiliary judgment can be made based on the corresponding 3D point cloud model to determine whether the steel strip segment has a defect. Specifically, the flatness of the steel strip segment can be detected using the acquired 3D point cloud model to further assist in determining whether the steel strip segment has a defect.

[0042] Step S4: Based on the position of each defect area in the corresponding steel strip slitting image, outline each defect area to obtain a defect illustration. Then, combine each defect illustration with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image and display it to the staff through the human-computer interaction display screen.

[0043] Specifically, based on the location of the defective area of ​​each steel strip in the steel strip slitting image, the defective area is delineated to obtain a defect illustration. Then, the defect illustration is combined with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image, and displayed to the worker through a human-machine interface display screen. The timely display of defect illustrations to the worker through the human-machine interface screen allows the worker to immediately understand the location of the defect.

[0044] More specifically, each defect is marked in different colors on the strip image according to its type. Furthermore, the dimensions of each defect area can be calculated based on the outline of each defect.

[0045] More preferably, this invention statistically analyzes the historical defect detection results of each steel strip slab, deduplicates and merges multiple defect areas of each steel strip slab to obtain the defect statistics for each steel strip slab, and then matches the statistical results of each steel strip slab with preset defect level rules to obtain the defect level and generate corresponding audible and visual alarm notifications based on the defect level. The preset defect level can be divided into fatal defects, major defects, and minor defects according to the actual degree of defect impact. Different defect levels have corresponding defect standards. For example, when the area of ​​a defective hole or deep scratch exceeds a certain number of square meters, the steel strip slab is considered to have a fatal defect. When proceeding to the next step, the steel strip slab needs to be promptly notified to the relevant maintenance personnel via audible and visual alarms so that corresponding measures can be taken in a timely manner. For example, when a minor scratch or pitting cluster (confidence level > 0.7, area > Y mm) is detected, the defect level is considered to be fatal. 2 When the current defect level is a major defect, a medium-level alarm is triggered.

[0046] More preferably, to ensure the quality of the acquired steel strip images, this invention also employs a multimodal fusion illumination system to supplement the camera's lighting. Specifically, when the camera captures images of multiple steel strips, the multimodal fusion illumination system is used to set different lighting modes, thereby acquiring steel strip images under different lighting modes. Specifically, the lighting modes include bright field illumination, dark field illumination, and coaxial illumination. Bright field illumination is more advantageous for detecting defects such as protrusions, pits, and dirt on the steel strips; dark field illumination achieves illumination through low-angle incident light, which is extremely sensitive to defects such as scratches and tiny burrs. Dark field illumination allows these tiny defects to be illuminated by scattered light, making it more advantageous for detecting scratches, burrs, and other tiny defects; coaxial illumination is mainly suitable for eliminating reflections and is more advantageous for detecting pits and holes on smooth surfaces. In other words, by setting up an illumination system that integrates multiple light sources in the testing company, the present invention can switch between different lighting modes in a very short time to take pictures of the steel strip strip at the same end to obtain multiple images of the steel strip under different lighting modes.

[0047] Furthermore, after obtaining the aforementioned multiple images of the steel strip under different lighting modes, this invention further extracts feature vectors from the steel strip images under different lighting modes to obtain feature maps of the steel strip images under different lighting modes. Then, the feature maps of the steel strip images under different lighting modes are fused to obtain a fused feature map, which is the steel strip image to be tested. In other words, this invention inputs the steel strip image of each lighting mode into an independent backbone branch to extract the most salient features under that lighting mode. Then, the feature maps of the same scale output from multiple branches are fused. During fusion, an intelligent weighted fusion method is used to assign appropriate weights to the spatial location, channel features, etc., of different branches to achieve the fusion of multiple feature maps.

[0048] More preferably, the present invention further preprocesses the steel strip images acquired under different lighting conditions, specifically including flattening correction, data augmentation, and normalization. Flattening correction refers to unifying the shadows in the background of the steel strip image caused by uneven lighting or lens vignetting to produce a background with uniform brightness. Data augmentation refers to removing noise from the corresponding steel strip images by simulating steel strip jitter, deviation, lighting fluctuations, and camera noise. Normalization refers to normalizing each steel strip image according to a preset image format and size. Simultaneously, preprocessing also ensures noise reduction and redundancy removal, guaranteeing the quality of the final steel strip image.

[0049] More preferably, the present invention further calculates the cumulative duration of defects for each steel strip slitting process. When the cumulative duration exceeds a preset time, a defect warning notification is generated based on the defect type, cumulative duration, start time, and end time of the corresponding steel strip slitting process, and fed back to the PLC industrial control computer of the steel strip slitting machine. This allows the PLC industrial control computer of the steel strip slitting machine to predict the cause of the defect based on the defect warning notification and an expert system built on an AI model, enabling dynamic fine-tuning of the equipment based on the predicted cause. If the number of fine-tuning attempts exceeds a preset number and the defect still exists in the corresponding steel strip slitting process, an audible and visual alarm notification or machine shutdown is triggered.

[0050] More preferably, when defects exist in the steel strip slitting, the present invention also obtains data signals collected by each sensor module of the steel strip slitting machine during operation from the system according to the defect start time and defect end time of each steel strip slitting, and analyzes the changing trend of the monitoring data collected by each sensor to determine whether there is an abnormality in the corresponding equipment. Then, based on the cause of the defect and the abnormal state of the equipment, the corresponding equipment is dynamically adjusted; and based on the cause of the defect, the steel strip slitting is dynamically fine-tuned according to the predicted analysis of the cause.

[0051] The sensor module includes tension sensors, vibration sensors, and acoustic emission sensors, used to acquire the tension of each frame of the strip slitting machine, the vibration spectrum of the cutter shaft and bearings, and the acoustic emission signal of cutter wear, respectively. Specifically: the tension sensor detects the real-time tension value of each frame of the strip slitting machine. To ensure the strip is properly divided, the frame is generally needed to assist in the smooth transport of the strip to the cutter position. Therefore, monitoring the real-time tension value determines whether the strip is transported smoothly within the slitting machine. The vibration sensor acquires the vibration spectrum of the cutter shaft and bearings. By analyzing the vibration spectrum of the cutter shaft and bearings, the vibration intensity of the cutter can be determined, thus revealing the relationship between the vibration intensity of the cutter and strip defects. The acoustic emission sensor characterizes the acoustic emission signal of cutter wear.

[0052] When the cutting tool is worn or cannot cut properly, defects will form on the steel strip slitting. Simultaneously, the detection data from the aforementioned tension sensor, vibration sensor, and acoustic emission sensor will show significant abnormal trends. Therefore, when a defect is detected in the currently output steel strip slitting, the detection data from these sensors can be retrieved from the system, and their changing trends analyzed. This allows for the identification of abnormal equipment, such as tool wear or insufficient tension, assisting in the location of problematic equipment and timely detection. Furthermore, expert systems or historical experience can be combined to dynamically adjust abnormal equipment. For example, if insufficient tension is detected by the tension sensor, tension control can be improved. If the defect disappears after dynamic feedback adjustment, no downtime is required, thus improving production efficiency. Additionally, when collecting the above data, this invention requires a unified clock source, such as a global trigger signal generated by an encoder, to synchronize the clocks of various systems or devices within the system. This ensures a one-to-one correspondence between the acquired image data and the collected sensor data, thereby achieving abnormal location of the steel strip slitting. Simultaneously, after the PLC industrial control computer collects the data, it sets a timestamp for the data collection time and saves the timestamp synchronously to the data. In this way, the timestamp and the collection time of the steel strip image can be matched to determine which sensor data corresponds to the cutting time of the steel strip image, so as to find the corresponding sensor data in a timely manner based on the steel strip slitting anomalies.

[0053] Example 2

[0054] The image-based real-time online inspection device for steel strip slitting includes a memory and a processor. The memory stores a point cloud stitching program that runs on the processor. The point cloud stitching program is a computer program. When the processor executes the point cloud stitching program, it performs the following steps:

[0055] Image acquisition steps: Periodically photograph the steel strips output by the steel strip slitting machine to obtain images of the steel strip to be tested; the images of the steel strip to be tested include multiple steel strips output by the steel strip slitting machine;

[0056] Defect detection steps: Based on the image of the steel strip to be tested and the defect detection model constructed by the system, it is determined whether there are defects in the steel strip slabs in the image of the steel strip to be tested. When there are defects in the steel strip slabs in the image of the steel strip to be tested, the steel strip slabs associated with each defect area are obtained according to the position of the defect area in the image of the steel strip to be tested. Then, the image of the steel strip to be tested is segmented according to the steel strip slabs with defects to obtain the steel strip slab image of each steel strip slab with defects.

[0057] Defect identification steps: Match each segmented steel strip image with the defect identification model built in the system to identify the defect type of each steel strip and the location of each defect region in the steel strip image;

[0058] Display steps: Based on the position of each defect area in the corresponding steel strip slitting image, outline each defect area to obtain a defect illustration. Then, combine each defect illustration with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image and display it to the staff through the human-computer interaction display screen.

[0059] Furthermore, the step of marking each defect illustration in the corresponding steel strip slicing image specifically includes: marking each defect illustration in the corresponding steel strip slicing image with different colors according to the defect type in each defect illustration; and calculating the size data of each defect area based on the outline of each defect.

[0060] Furthermore, the display steps also include: statistically analyzing the historical defect detection results of each steel strip slab, deduplicating and merging multiple defect areas of each steel strip slab to obtain the defect statistics of each steel strip slab, then matching the statistical results of each steel strip slab with preset defect level rules to obtain the defect level, and generating corresponding audible and visual alarm notifications based on the defect level.

[0061] Furthermore, the construction process of the defect detection model specifically includes: preprocessing each historical steel strip image that does not contain defects by statistically analyzing historical images of the steel strip to be tested in the system; extracting feature vectors from each historical steel strip image, then constructing a model based on a CNN detection network model and training the model based on the historical steel strip images to obtain the defect detection model; and dynamically updating the defect detection model constructed in the system based on newly acquired steel strip images.

[0062] Furthermore, the construction process of the defect type identification model specifically includes: acquiring multiple steel strip images of each defect type and preprocessing each steel strip image; dividing each steel strip image into grids to obtain multiple grids, and extracting features from each grid of each steel strip image based on the features of each defect type of the steel strip to obtain the feature vector of each steel strip image; then training the feature vector of each steel strip image based on a convolutional neural network model to construct a set of feature vectors of multiple steel strip images of each defect type; and constructing a defect identification model for different defect types based on the combination of the feature vectors of multiple steel strip images of each defect type.

[0063] Furthermore, image data acquisition also includes: setting up a multimodal fusion lighting system at the process inspection point to provide different illuminations to supplement the camera, thereby acquiring steel strip images under different illumination modes; and extracting feature vectors from the steel strip images under different illumination modes to obtain feature maps of the steel strip images under different illumination modes, and fusing the feature maps of the steel strip images under different illumination modes to obtain the steel strip image to be tested; wherein, the illumination modes include: bright field illumination, dark field illumination and coaxial light illumination.

[0064] Furthermore, the image data acquisition steps also include: preprocessing the steel strip images acquired under different lighting modes; wherein, the preprocessing includes flattening correction, data augmentation, and normalization processing; wherein, flattening correction refers to unifying the shadows in the background of the steel strip image caused by uneven lighting or lens vignetting to produce a background with uniform brightness; data augmentation refers to removing noise from the corresponding steel strip image by simulating the jitter, deviation, lighting fluctuations, and camera noise of the steel strip; normalization processing refers to normalizing each steel strip image according to a preset image format and size.

[0065] Furthermore, when the processor executes the point cloud stitching program, it implements the following steps: Early warning step: When the cumulative time of the corresponding steel strip slitting defect exceeds the preset time, a defect early warning notification is generated based on the defect type, cumulative duration of the defect, defect start time, and defect end time of the corresponding steel strip slitting and fed back to the PLC industrial control computer of the steel strip slitting machine, so that the PLC industrial control computer of the steel strip slitting machine can predict the cause of the defect based on the defect early warning notification and the expert system built based on the AI ​​model;

[0066] Simultaneously, based on the defect start time and defect end time of each steel strip slitting, the system acquires data signals collected by various sensor modules during the operation of the steel strip slitting machine. The system analyzes the changing trends of the monitoring data collected by each sensor to determine if there are any abnormalities in the corresponding equipment. Then, based on the cause of the defect and the abnormal state of the equipment, the system dynamically adjusts the corresponding equipment. Furthermore, based on the cause of the defect, the system dynamically fine-tunes the steel strip slitting process according to the predicted analysis. When the number of fine-tuning operations exceeds a preset number and the corresponding steel strip slitting defect still exists, an audible and visual alarm is issued or the machine is stopped. The sensor modules include tension sensors, vibration sensors, and acoustic emission sensors, which are used to acquire the tension of each frame of the steel strip slitting machine, the vibration spectrum of the cutter shaft and bearings, and the acoustic emission signals of cutter wear, respectively.

[0067] Example 3

[0068] A computer-readable storage medium storing a real-time online detection program for steel strip slitting, the steel strip real-time online detection program being a computer program, which, when executed by a processor, performs the following steps:

[0069] Image acquisition steps: Periodically photograph the steel strips output by the steel strip slitting machine to obtain images of the steel strip to be tested; the images of the steel strip to be tested include multiple steel strips output by the steel strip slitting machine;

[0070] Defect detection steps: Based on the image of the steel strip to be tested and the defect detection model constructed by the system, it is determined whether there are defects in the steel strip slabs in the image of the steel strip to be tested. When there are defects in the steel strip slabs in the image of the steel strip to be tested, the steel strip slabs associated with each defect area are obtained according to the position of the defect area in the image of the steel strip to be tested. Then, the image of the steel strip to be tested is segmented according to the steel strip slabs with defects to obtain the steel strip slab image of each steel strip slab with defects.

[0071] Defect identification steps: Match each segmented steel strip image with the defect identification model built in the system to identify the defect type of each steel strip and the location of each defect region in the steel strip image;

[0072] Display steps: Based on the position of each defect area in the corresponding steel strip slitting image, outline each defect area to obtain a defect illustration. Then, combine each defect illustration with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image and display it to the staff through the human-computer interaction display screen.

[0073] Furthermore, the step of marking each defect illustration in the corresponding steel strip slicing image specifically includes: marking each defect illustration in the corresponding steel strip slicing image with different colors according to the defect type in each defect illustration; and calculating the size data of each defect area based on the outline of each defect.

[0074] Furthermore, the display steps also include: statistically analyzing the historical defect detection results of each steel strip slab, deduplicating and merging multiple defect areas of each steel strip slab to obtain the defect statistics of each steel strip slab, then matching the statistical results of each steel strip slab with preset defect level rules to obtain the defect level, and generating corresponding audible and visual alarm notifications based on the defect level.

[0075] Furthermore, the construction process of the defect detection model specifically includes: preprocessing each historical steel strip image that does not contain defects by statistically analyzing historical images of the steel strip to be tested in the system; extracting feature vectors from each historical steel strip image, then constructing a model based on a CNN detection network model and training the model based on the historical steel strip images to obtain the defect detection model; and dynamically updating the defect detection model constructed in the system based on newly acquired steel strip images.

[0076] Furthermore, the construction process of the defect type identification model specifically includes: acquiring multiple steel strip images of each defect type and preprocessing each steel strip image; dividing each steel strip image into grids to obtain multiple grids, and extracting features from each grid of each steel strip image based on the features of each defect type of the steel strip to obtain the feature vector of each steel strip image; then training the feature vector of each steel strip image based on a convolutional neural network model to construct a set of feature vectors of multiple steel strip images of each defect type; and constructing a defect identification model for different defect types based on the combination of the feature vectors of multiple steel strip images of each defect type.

[0077] Furthermore, image data acquisition also includes: setting up a multimodal fusion lighting system at the process inspection point to provide different illuminations to supplement the camera, thereby acquiring steel strip images under different illumination modes; and extracting feature vectors from the steel strip images under different illumination modes to obtain feature maps of the steel strip images under different illumination modes, and fusing the feature maps of the steel strip images under different illumination modes to obtain the steel strip image to be tested; wherein, the illumination modes include: bright field illumination, dark field illumination and coaxial light illumination.

[0078] Furthermore, the image data acquisition steps also include: preprocessing the steel strip images acquired under different lighting modes; wherein, the preprocessing includes flattening correction, data augmentation, and normalization processing; wherein, flattening correction refers to unifying the shadows in the background of the steel strip image caused by uneven lighting or lens vignetting to produce a background with uniform brightness; data augmentation refers to removing noise from the corresponding steel strip image by simulating the jitter, deviation, lighting fluctuations, and camera noise of the steel strip; normalization processing refers to normalizing each steel strip image according to a preset image format and size.

[0079] Furthermore, when the real-time online detection program for steel strip slitting is executed by the processor, it implements the following steps: Early warning step: When the cumulative time of defects in the corresponding steel strip exceeds the preset time, a defect early warning notification is generated based on the defect type, cumulative duration of defects, defect start time, and defect end time of the corresponding steel strip and fed back to the PLC industrial control computer of the steel strip slitting machine, so that the PLC industrial control computer of the steel strip slitting machine can predict the cause of defect formation based on the defect early warning notification and the expert system built based on the AI ​​model;

[0080] Simultaneously, based on the defect start time and defect end time of each steel strip slitting, the system acquires data signals collected by various sensor modules during the operation of the steel strip slitting machine. The system analyzes the changing trends of the monitoring data collected by each sensor to determine if there are any abnormalities in the corresponding equipment. Then, based on the cause of the defect and the abnormal state of the equipment, the system dynamically adjusts the corresponding equipment. Furthermore, based on the cause of the defect, the system dynamically fine-tunes the steel strip slitting process according to the predicted analysis. When the number of fine-tuning operations exceeds a preset number and the corresponding steel strip slitting defect still exists, an audible and visual alarm is issued or the machine is stopped. The sensor modules include tension sensors, vibration sensors, and acoustic emission sensors, which are used to acquire the tension of each frame of the steel strip slitting machine, the vibration spectrum of the cutter shaft and bearings, and the acoustic emission signals of cutter wear, respectively.

[0081] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A real-time online detection method for steel strip slitting based on images, characterized in that, The real-time online detection method for steel strip slitting includes: Image acquisition steps: Periodically photograph the steel strip slits output by the steel strip slitting machine to obtain images of the steel strip to be tested; the images of the steel strip to be tested include multiple steel strip slits output by the steel strip slitting machine; Defect detection steps: Based on the image of the steel strip to be tested and the defect detection model constructed by the system, it is determined whether there are defects in the steel strip slits in the image of the steel strip to be tested. When there are defects in the steel strip slits in the image of the steel strip to be tested, the steel strip slits associated with each defect area are obtained according to the position of the defect area in the image of the steel strip to be tested. Then, the image of the steel strip to be tested is segmented according to the steel strip slits with defects to obtain the steel strip slit image of each steel strip slit with defects. Defect identification steps: Match each segmented steel strip image with the defect identification model built in the system to identify the defect type of each steel strip and the location of each defect region in the steel strip image; Display steps: Based on the position of each defect area in the corresponding steel strip slitting image, the outline of each defect area is drawn to obtain a defect illustration. Then, each defect illustration is combined with the corresponding steel strip slitting image to mark each defect illustration in the corresponding steel strip slitting image and displayed to the staff through a human-computer interaction display screen.

2. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, The display step of marking each defect image in the corresponding steel strip slicing image specifically includes: marking each defect image in the corresponding steel strip slicing image with different colors according to the defect type in each defect image; and calculating the size data of each defect area based on the outline of each defect.

3. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, The display steps further include: statistically analyzing the historical defect detection results of each steel strip and deduplicating and merging multiple defect areas of each steel strip to obtain the defect statistics of each steel strip; then matching the statistical results of each steel strip with preset defect level rules to obtain the defect level and generating corresponding audible and visual alarm notifications based on the defect level.

4. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, The construction process of the defect detection model specifically includes: preprocessing each historical steel strip image that does not have defects by statistically analyzing historical images of the steel strip to be tested that do not have defects; extracting feature vectors from each historical steel strip image, then constructing a model based on a CNN detection network model and training the model based on the historical steel strip images to obtain the defect detection model; and dynamically updating the defect detection model constructed in the system based on newly acquired steel strip images.

5. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, The construction process of the defect type identification model specifically includes: acquiring multiple steel strip images of each defect type and preprocessing each steel strip image; dividing each steel strip image into grids to obtain multiple grids, and extracting features from each grid of each steel strip image based on the features of each defect type of the steel strip to obtain the feature vector of each steel strip image; then training the feature vector of each steel strip image based on a convolutional neural network model to construct a set of feature vectors of multiple steel strip images of each defect type; and constructing a defect identification model for different defect types based on the combination of the feature vectors of multiple steel strip images of each defect type.

6. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, The image data acquisition further includes: setting up a multimodal fusion lighting system at the process inspection point to provide different illuminations to supplement the camera, thereby acquiring steel strip images under different illumination modes; and extracting feature vectors from the steel strip images under different illumination modes to obtain feature maps of the steel strip images under different illumination modes, and fusing the feature maps of the steel strip images under different illumination modes to obtain the steel strip image to be tested; wherein, the illumination modes include: bright field illumination, dark field illumination and coaxial light illumination.

7. The image-based real-time online detection method for steel strip slitting according to claim 6, characterized in that, The image data acquisition step further includes: preprocessing the acquired steel strip images under different lighting modes; wherein, the preprocessing includes flattening correction, data augmentation, and normalization; wherein, flattening correction refers to unifying the shadows in the background of the steel strip image caused by uneven lighting or lens vignetting to produce a background with uniform brightness; the data augmentation refers to removing noise from the corresponding steel strip image by simulating the jitter, deviation, lighting fluctuation, and camera noise of the steel strip; the normalization process refers to normalizing each steel strip image according to a preset image format and size.

8. The image-based real-time online detection method for steel strip slitting according to claim 1, characterized in that, It also includes an early warning step: when the cumulative time of defects in the corresponding steel strip exceeds a preset time, a defect early warning notification is generated based on the defect type, cumulative duration of defects, defect start time, and defect end time of the corresponding steel strip and fed back to the PLC industrial control computer of the steel strip slitting machine, so that the PLC industrial control computer of the steel strip slitting machine can predict the cause of defect formation based on the defect early warning notification and the expert system built based on the AI ​​model; Simultaneously, based on the defect start time and defect end time of each steel strip slitting, the system acquires data signals collected by various sensor modules during the operation of the steel strip slitting machine. The system analyzes the changing trends of the monitoring data collected by each sensor to determine if there are any abnormalities in the corresponding equipment. Then, based on the cause of the defect and the abnormal state of the equipment, the system dynamically adjusts the corresponding equipment. Furthermore, based on the cause of the defect, the system dynamically fine-tunes the steel strip slitting process according to the predicted analysis. When the number of fine-tuning operations exceeds a preset number and the corresponding steel strip slitting defect still exists, an audible and visual alarm is issued or the machine is stopped. The sensor modules include tension sensors, vibration sensors, and acoustic emission sensors, which are used to acquire the tension of each frame of the steel strip slitting machine, the vibration spectrum of the cutter shaft and bearings, and the acoustic emission signals of cutter wear, respectively.

9. An image-based real-time online inspection device for steel strip slitting, comprising a memory and a processor, wherein the memory stores a point cloud stitching program running on the processor, the point cloud stitching program being a computer program, characterized in that, When the processor executes the point cloud stitching program, it implements the steps of a point cloud stitching method for house types as described in any one of claims 1-8.

10. A computer-readable storage medium storing thereon a real-time online detection program for steel strip slitting, characterized in that, The real-time online detection program for steel strip slitting is a computer program. When the real-time online detection program for steel strip slitting is executed by a processor, it implements the steps of the image-based real-time online detection method for steel strip slitting as described in any one of claims 1-8.

Images