A flying steel detection method, system, storage medium and program product

Abstract

The present invention provides a flying steel detection method, system, storage medium and program product, wherein the method includes: acquiring real-time video stream image data; using a target detection model to detect flying steel targets; using image gray value statistical features and classification models to perform comprehensive identification of flying steel targets in dense fog; performing comprehensive judgment based on the flying steel target detection results and the comprehensive identification results of flying steel targets in dense fog; when the comprehensive judgment result is that flying steel is occurring, sending an alarm to the server side, turning on the flying shear device to cut it, and performing log and data recording; performing user feedback and model incremental learning. The present invention can improve the detection accuracy of flying steel, reduce the system misjudgment rate, increase the robustness of the algorithm, and improve the adaptability of the model to scene changes. In an actual production environment, it can reduce the production safety accident rate, reduce the time and production cost losses caused by flying steel, maintain production efficiency and benefits, and improve energy utilization.

Description

Technical Field

[0001] The present invention relates to the field of high-speed wire steel production automation, and in particular to a flying steel detection method, system, storage medium and program product. Background Art

[0002] High-speed wire production is a highly automated metal forming process that involves rolling hot steel billets through a series of rolling mills to form wires of various diameters. In steel production, high-speed jets cause molten steel to fly out in the air in long, scattered strips, forming flywires. Flywires are a common and serious production accident in the wire rolling process, which can cause production interruptions and even equipment damage, increasing material losses and reducing production efficiency, and can also lead to serious safety issues. Flywires can be attributed to a variety of factors, such as mechanical failures of rolling mills, sensor failures or control system errors, irregular billet shapes, internal defects or uneven temperatures, improper settings of rolling speed, temperature, tension or cooling conditions, etc. Detecting and promptly handling flywires can effectively reduce losses and safety accident rates.

[0003] There are several types of existing flying steel detection algorithms. One type is based on manual observation, rolling signals or multiple sensing devices. Manual observation may have the problem of slow response and untimely operation. The acquisition of rolling signals requires production line transformation, and in the actual production environment, rolling signals cannot absolutely determine whether flying steel has occurred. Sensing equipment such as thermal imaging is sensitive to on-site environmental factors and cannot be stably detected. One type is based on machine vision and deep learning methods, such as traditional image processing methods such as binarization and threshold segmentation, as well as deep learning methods such as classification, detection, and segmentation. Traditional image processing methods require complex parameter adjustment processes and are difficult to adapt to the ever-changing production environment. Existing deep learning methods lack a systematic treatment method for model misdetection, which is prone to misjudgment, affecting the production rhythm and causing economic losses. At the same time, the steel production site is often surrounded by fog. When the fog is thick, methods based on detection and segmentation may not be able to detect flying steel targets. In addition, in the face of complex and changing production environments, there is a lack of long-term model automatic update mechanism, which is easy to lead to continuous deterioration of detection results and more false detections.

[0004] In summary, the existing flying steel detection methods have the problems of poor handling and control capabilities for false detection, inability to detect in dense fog environments, and difficulty in adapting models to scene changes. Therefore, the development of flying steel detection systems and methods still faces challenges and urgently needs to be optimized in order to achieve rapid detection and response to flying steel events that occur during high-speed wire production. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a flying steel detection method, system, storage medium and program product, which aim to process false detections by introducing scene rule knowledge and empirical knowledge of flying steel movement patterns, thereby achieving effective control of misjudgments; and adding a flying steel recognition module in a dense fog environment to improve the flying steel recognition rate; and then based on the detection data records and user feedback, automatic false detection data screening, labeling and model incremental updating are performed, so that the model can adapt to the ever-changing on-site environment, control false detections and improve model robustness.

[0006] In a first aspect, the present invention provides a method for detecting a flying steel, comprising the following steps:

[0007] Get real-time video stream image data;

[0008] Use the target detection model to detect flying steel targets;

[0009] The image gray value statistical features and classification model are used to comprehensively identify flying steel targets in dense fog;

[0010] Comprehensive judgment is made based on the flying steel target detection results and the comprehensive identification results of flying steel targets in dense fog;

[0011] When the comprehensive judgment result is that flying steel is occurring, an alarm is sent to the server, the flying shear device is turned on to cut, and logs and data are recorded;

[0012] Conduct user feedback and model incremental learning.

[0013] As a further improvement of the present invention, the method of using the target detection model to perform flying steel target detection includes:

[0014] Use the target detection model to infer the image data and obtain the inference results;

[0015] Based on scene rules and motion patterns, the non-real flying steel targets in the inference results are filtered out to obtain the detection results;

[0016] Based on the detection results of multiple consecutive frames of images, the flying steel process is determined.

[0017] As a further improvement of the present invention, the training method of the target detection model includes:

[0018] Collect Feigang target image data;

[0019] The flying steel targets are manually labeled as training data to train the target detection model.

[0020] As a further improvement of the present invention, the filtering of non-real flying steel targets in the inference results based on scene rules and motion patterns includes:

[0021] Adopting scene rules, based on the flow area of ​​steel and the coverage of actual flying steel, the region of interest is divided and the detection targets outside the region of interest are screened out;

[0022] Identify the motion pattern of the detection target based on the position and size changes of the detection frame, and screen out the detection targets that do not conform to the Feigang motion pattern.

[0023] As a further improvement of the present invention, the method for determining the flying steel process is as follows: setting a determination time and a determination number, when the number of flying steel targets appearing within the determination time is above the determination number, it is determined to be true, that is, flying steel is occurring on site.

[0024] As a further improvement of the present invention, the method of using image gray value statistical features and classification models to comprehensively identify flying steel targets in dense fog includes:

[0025] Calculate the statistical features of the grayscale value of the image, perform statistical analysis based on the feature values ​​of the image of flying steel in dense fog, the image without flying steel in dense fog, and the image in non-dense fog scenes, set the threshold of each feature dimension, and set comprehensive judgment rules to identify the flying steel in dense fog during real-time detection;

[0026] Build binary classification models for flying steel in dense fog and other scenes, infer the real-time collected images, identify whether they are flying steel in dense fog, and enhance samples through HSV transformation and flying steel halo size, position, and shape transformation to train the binary classification model;

[0027] A comprehensive judgment is made on whether flying steel occurs in dense fog. When the characteristic value satisfies the comprehensive judgment rules and the classification confidence exceeds the preset threshold, it is judged to be true, that is, flying steel is occurring on site.

[0028] As a further improvement of the present invention, the comprehensive determination based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog includes:

[0029] If either the flying steel target detection result or the flying steel target comprehensive identification result under dense fog is judged to be true, it is comprehensively judged that flying steel is occurring at the scene.

[0030] As a further improvement of the present invention, the user feedback includes:

[0031] When an alarm record is a real flying steel, the feedback is a real flying steel;

[0032] When no flying steel error alarm occurs, the feedback is a false alarm;

[0033] When flying steel occurs on site and there is no alarm record on the client side, the user manually enters the missed alarm information.

[0034] As a further improvement of the present invention, the model incremental learning includes:

[0035] Obtain false detection data based on user feedback;

[0036] Clean the false detection data, missed data, and data within the production line distance threshold to obtain an incremental data set;

[0037] Use the detection model to infer the incremental data set and generate pseudo labels;

[0038] When the production line is stopped, GPU resources are called to perform incremental model training and updates based on incremental data sets.

[0039] In a second aspect, the present invention provides a computer system, which is used to implement the steps of the method described in the first aspect, including a video stream image frame acquisition module, a flying steel target determination module, a dense fog scene flying steel determination module, a comprehensive determination module, a real-time processing module, and a user feedback and incremental learning module;

[0040] The video stream image frame acquisition module is used to obtain real-time video stream image data;

[0041] The flying steel target determination module is used to perform flying steel target detection using a target detection model;

[0042] The dense fog scene flying steel determination module is used to use the image gray value statistical characteristics and classification model to perform comprehensive recognition of flying steel targets in dense fog;

[0043] The comprehensive judgment module is used to make a comprehensive judgment based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog;

[0044] The real-time processing module is used to send an alarm to the server when the comprehensive judgment result shows that flying steel is occurring, start the flying shear device to cut, and record logs and data;

[0045] The user feedback and incremental learning module is used to perform user feedback and model incremental learning.

[0046] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0047] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention realizes fast and accurate flying steel target detection by using a detection model, reduces the time delay from the occurrence of flying steel to the detection of flying steel and the issuance of an alarm, and improves the detection efficiency.

[0050] Second, light changes and floating fog in the production environment may cause frequent misdetection of on-site equipment and some areas, and even generate false alarms, leading to production accidents. The present invention introduces scene rule knowledge and flying steel motion pattern experience knowledge to deal with false detection, thereby achieving effective control of false judgments, effectively reducing the probability of false judgments caused by false detections, and enhancing the stability of the detection system;

[0051] 3. The fog in some on-site production environments is so heavy that flying steel may not be detected when it occurs. The present invention adds a flying steel recognition module in a dense fog environment. By calculating the statistical features of the image grayscale value and training a binary classification model, the image of flying steel in dense fog is comprehensively recognized, thereby enhancing the flying steel detection capability in dense fog scenes and improving the flying steel recognition rate.

[0052] 4. Based on detection data records and user feedback, automatic false detection data screening, labeling and model incremental updates are carried out to enable the model to continuously adapt to changes in image features brought about by scene changes, maintain a low false detection rate and improve model stability.

[0053] 5. By combining user feedback, data screening, data labeling, and model incremental updates can be automatically implemented, which can reduce the maintenance cost of the detection model and reduce manual workload.

[0054] 6. Through fast and accurate flying steel detection, judgment and alarm, further linkage with on-site PLC and flying shear equipment can achieve rapid shearing and production stop when flying steel occurs, avoiding the continued increase of flying steel coverage area and injuring workers, reducing safety accident rate and economic losses.

[0055] In summary, the present invention can improve the detection accuracy of flying steel, reduce the system misjudgment rate, increase the robustness of the algorithm, and improve the model's adaptability to scene changes. In the actual production environment, it can reduce the rate of production safety accidents, reduce the time and production costs caused by flying steel, maintain production efficiency and benefits, and improve energy utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flow chart of a flying steel detection method provided by an embodiment of the present invention;

[0057] Figure 2 A flowchart of user feedback and model incremental learning provided by an embodiment of the present invention;

[0058] Figure 3 A framework diagram of a flying steel detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the present invention will combine the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, wherein steps S1, S2... in the embodiments described in the present invention do not limit the only execution steps of the present invention; the various models described in the present invention are not the only limiting methods of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] In the present invention, computer devices / equipment / systems refer to related entities applied to computers, such as hardware, a combination of hardware and software, software or software in execution, etc. In detail, for example, software includes but is not limited to a process running on a processor, a processor, an object, executable software, an execution thread, a program and / or a computer. In addition, an application or script program running on a server, a server can also be software. One or more software can be in an execution process and / or thread, and the software can be localized on one computer and / or distributed between two or more computers, and can be run by various computer-readable media.

[0061] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0062] In a first aspect, a method for detecting flying steel is provided in an embodiment of the present invention. Figure 1 As shown, the specific steps can be as follows:

[0063] S100: Acquire real-time video stream image data.

[0064] In one embodiment of the present invention, a high-speed, high-resolution and explosion-proof industrial camera is installed in the on-site monitoring area to obtain real-time, high-definition on-site images and prevent flying steel from hitting the camera and causing damage; the camera position is adjusted to cover the monitored area. For example, each camera is set to monitor the range of a rolling mill, and appropriate focal length adjustments are made to ensure that the camera can clearly capture the monitored area. Based on the industrial cameras deployed on-site, real-time video stream data of the camera can be obtained through network connections and camera development tools. According to the detection speed and algorithm requirements, frames are extracted from it to obtain real-time image data as input to the detection algorithm.

[0065] S200: Use the target detection model to perform flying steel target detection;

[0066] Preferably, the present invention uses the yolov8 detection model to detect flying steel targets, which can achieve a good balance between reasoning speed and accuracy.

[0067] Preferably, the use of the target detection model to perform flying steel target detection includes:

[0068] S210: Use the target detection model to infer the image data to obtain an inference result.

[0069] Preferably, the training method of the target detection model includes:

[0070] S211: Collect Feigang target image data;

[0071] Before the detection system is put into operation, it is necessary to collect on-site image data over a period of time, mainly images when flying steel occurs.

[0072] S212: Manually label the flying steel targets as training data to train the target detection model.

[0073] In one embodiment of the present invention, the yolov8 detection model is used for target detection. Before the detection system is run, it is necessary to collect on-site image data within a period of time, mainly images when flying steel occurs, and manually mark the flying steel targets as training data to train the yolov8 detection model. When the detection system is running, after obtaining real-time image data, the model is used for reasoning to obtain the reasoning result.

[0074] S220: filtering non-real flying steel targets in the inference results based on scene rules and motion patterns to obtain detection results;

[0075] The model may have false detections, detecting other targets as flying steel, especially at steel production sites, where light and illumination are constantly changing and there is a lot of water mist, which increases the risk of false detection. Therefore, it is necessary to further filter targets based on the model detection results to screen out false flying steel targets.

[0076] Preferably, the filtering of non-real flying steel targets in the inference results based on scene rules and motion patterns includes:

[0077] S221: Using scene rules, based on the flow area of ​​steel and the coverage of actual flying steel, the region of interest is divided and the detection targets outside the region of interest are screened out;

[0078] S222: Identify the motion pattern of the detection target according to the position and size changes of the detection frame, and filter out the detection targets that do not conform to the flying steel motion pattern.

[0079] In one embodiment of the present invention, firstly, scene rules are adopted, based on the layout of on-site equipment, the position of the production line, etc., that is, based on the flow area of ​​steel, according to the coverage of the actual flying steel, the region of interest (ROI) is divided, and the detection targets outside the ROI are screened out. This step can limit the occurrence and detection position of flying steel, reduce the algorithm's focus range, and increase the accuracy; secondly, the motion mode of the detection target is identified according to the position and size change of the detection frame. Specifically, the detection target can be identified by calculating the position and size change of the flying steel target in several consecutive frames. Whether the detection target is in a static mode or a motion mode, and identifying the change mode of the detection target continuously moving left / right, irregular movement, etc., screen out the detection targets that do not conform to the flying steel motion mode, and filter out false flying steel targets. The detection targets that do not conform to the flying steel motion mode may include targets with very small changes in position and size, targets with very small changes in size and positions that are constantly moving in a certain direction, etc. Targets with very small changes in position and size can be considered as on-site equipment that is mistakenly detected as flying steel, and targets with very small changes in size and positions that are constantly moving in a certain direction can be considered as on-site hanging objects or people carrying objects.

[0080] Through the above-mentioned scene rules and motion pattern filtering, the probability of system misjudgment due to model misdetection and the resulting losses can be effectively reduced.

[0081] S230: Based on the detection results of the continuous multi-frame images, the flying steel process is determined.

[0082] Preferably, the method for determining the flying steel process is as follows: setting a determination time and a determination number, and when the number of flying steel targets appearing within the determination time is above the determination number, the determination is true, that is, flying steel is occurring on site.

[0083] In one embodiment of the present invention, when a flying steel target is detected 3 times or more within a period of 5 consecutive detection frames, it is determined to be true, that is, flying steel is occurring at the scene.

[0084] It should be noted that, in addition to preferably using the yolov8 detection model of the present invention, flying steel target detection can also use traditional image processing methods and deep learning segmentation methods. Traditional image processing methods extract the red steel part through binarization, connected domain extraction, corrosion, expansion and other operations, but this type of method requires complex parameter adjustment, and in the environment where the light and fog on the scene are constantly changing, the algorithm is difficult to continue to take effect; deep learning segmentation methods, such as SAM, can directly extract the red steel part based on the trained model, but the annotation work of the segmentation model is complicated, and there is still a phenomenon of false detection.

[0085] S300: Comprehensive recognition of flying steel targets in dense fog using image grayscale statistical features and classification models;

[0086] The outline of flying steel is usually invisible in dense fog. The present invention is based on the special image features in dense fog scenes, combined with the gray value statistical features and the classification model for comprehensive judgment, which can improve the recognition rate of flying steel in dense fog.

[0087] Preferably, the method of using image gray value statistical features and classification models to perform comprehensive identification of flying steel targets in dense fog includes:

[0088] S310: Calculate the statistical features of the grayscale value of the image, perform statistical analysis based on the feature values ​​of the image of the flying steel in the dense fog, the feature values ​​of the image without flying steel in the dense fog, and the image in the non-dense fog scene, set the threshold of each feature dimension, and the comprehensive judgment rule, so as to identify the flying steel in the dense fog during real-time detection;

[0089] S320: construct binary classification models for flying steel in dense fog and other scenes, perform reasoning on the real-time collected images, identify whether they belong to flying steel in dense fog, and perform sample enhancement through HSV transformation and flying steel halo size, position, and shape transformation to train the binary classification model;

[0090] In one embodiment of the present invention, in the first aspect, in a dense fog scene, the image usually presents an off-white background, and a red-yellow halo appears when flying steel occurs. Based on this, the grayscale statistical features of the image are calculated and selected, including the average grayscale, maximum grayscale, minimum grayscale, grayscale variance, etc. of the three channels of R, G, and B. Based on the above-mentioned characteristic values ​​of the flying steel image under dense fog, and the characteristic values ​​of other scenes, including images in dense fog without flying steel and non-dense fog scenes, statistical analysis is performed to set the thresholds of each characteristic dimension, and a comprehensive judgment rule is used to identify the flying steel occurring in dense fog during real-time detection. For example, in a dense fog scene with an off-white background as the main background, the grayscale mean of the RGB three channels can be set within the threshold range of 60 to 200, and when flying steel occurs, the grayscale value of the R channel will increase significantly. It can be set that when the grayscale mean of the R channel is greater than the mean of the G and B channels by more than 40, it is considered that the flying steel has been identified in the dense fog. Secondly, a binary classification model of flying steel in dense fog and other scenes is constructed to identify flying steel. Since the amount of flying steel data in dense fog may be small, data enhancement is required. Sufficient samples are generated through HSV transformation, flying steel halo size, position, shape transformation, etc. First, the binary classification model is trained, and then online inference is performed to obtain the flying steel identification results.

[0091] S330: Comprehensively determine whether flying steel occurs in dense fog. When the characteristic value meets the comprehensive determination rule and the classification confidence exceeds the preset threshold, it is determined to be true, that is, flying steel is occurring on site.

[0092] Based on the recognition results obtained by the above two methods, a comprehensive judgment is made as to whether flying steel occurs in dense fog, that is, when the characteristic value in S310 satisfies the judgment rule and the classification confidence of the method in S320 exceeds the preset threshold, it is judged to be true, that is, flying steel is occurring on site.

[0093] It should be noted that, in addition to the image gray value statistical features and classification model comprehensive recognition method described in the present invention, hot metal detectors and other detection equipment can also be used to identify flying steel in dense fog. When the fog is heavy and it is difficult to detect clear flying steel targets in the image, the use of thermal detection methods can identify the movement of hot metal and assist in the identification of flying steel. However, the occurrence of flying steel is changeable and the movement pattern is irregular. It is difficult to ensure that the detection equipment can effectively capture the flying steel, which is easy to cause missed detection.

[0094] S400: performing comprehensive judgment based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog;

[0095] The comprehensive determination based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog includes:

[0096] If either the flying steel target detection result or the flying steel target comprehensive identification result under dense fog is judged to be true, it is comprehensively judged that flying steel is occurring on site, and the subsequent alarm and other processing procedures are entered.

[0097] S500: When the comprehensive judgment result is that flying steel is occurring, an alarm is sent to the server, the flying shear device is turned on to cut, and logs and data are recorded;

[0098] In one embodiment of the present invention, the detection system includes a server side that runs the algorithm and stores data, and a client side that displays alarm information to the user and provides an operation interface. When it is detected that flying steel is occurring, firstly, an alarm is firstly sent to the server side, an alarm record is generated on the client side, and it is linked with the PLC on site. When flying steel occurs, the alarm light is activated, and the flying shear device is turned on to cut it off to prevent the flying steel from continuing to expand its range; secondly, in addition to generating the client alarm record, the server side also records the operation process and judgment results of the algorithm. In addition, when the model detects a flying steel target, the detection image and results are saved and used as incremental data to participate in the incremental update of the model.

[0099] In another embodiment of the present invention, when it is detected that flying steel is occurring, in addition to being linked with the on-site PLC, when flying steel occurs, the alarm light is activated, and the flying shear device is opened for cutting, the worker can also manually open the flying shear device for cutting after hearing the alarm.

[0100] S600: Figure 2 As shown, user feedback and model incremental learning are performed.

[0101] Among them, model incremental learning means that the model continuously learns new data without the need to start training from scratch, so that the model can continuously adapt to changes in data and scenarios.

[0102] Preferably, the user feedback includes:

[0103] When an alarm record is a real flying steel, the feedback is a real flying steel;

[0104] When no flying steel error alarm occurs, the feedback is a false alarm;

[0105] When flying steel occurs on site and there is no alarm record on the client side, the user manually enters the missed alarm information.

[0106] In one embodiment of the present invention, a user feedback function is added to the client interface. During the operation of the system, the user provides feedback on the correctness of the flying steel alarm issued by the detection system to assist the automatic update of the model. When an alarm record is a real flying steel, the feedback is a real flying steel; when no flying steel error alarm occurs, the feedback is a false alarm; when flying steel occurs on site and there is no alarm record on the client, the user can manually enter the missed alarm information, which includes the time period and location of occurrence.

[0107] Preferably, the incremental model learning refers to collecting and cleaning data from the stored false positive data based on user feedback to obtain a phased false positive incremental data set, and based on the data set, realizing automatic incremental model learning and model updating, including:

[0108] S610: Acquire false detection data according to user feedback;

[0109] In one embodiment of the present invention, data collection is performed: based on user feedback, the false detection data saved during the system operation is obtained, especially the data within the time period of the false alarm record reported by the user, which is the serious false detection data; the false detection but non-false alarm data is also collected.

[0110] S620: Cleaning the flying steel data, missed data and data within the production line distance threshold in the false detection data to obtain an incremental data set;

[0111] In one embodiment of the present invention, data cleaning is performed: based on user feedback, firstly, the data in the time period when flying steel actually occurs and the data in the time period when underreporting occurs are screened out. This part of the data is the data when flying steel actually occurs, and needs to be manually verified and annotated and corrected before being added to the model training set; in addition, based on the position information of the detection frame, the false detection data within the distance threshold between the detection frame on the image and the production line is screened out. The specific distance threshold needs to be adjusted according to the image resolution and the deployment of on-site equipment. For example, the false detection data within 50 pixels of the detection frame on the image from the production line is screened out. This part of the data may be false detection caused by slight arching when steel passes through, and other normal production phenomena, and is not used as false detection data for the model update data set. The false detection data set after cleaning is used as a phased incremental data set.

[0112] S630: Use the detection model to infer the incremental data set and generate pseudo labels;

[0113] In one embodiment of the present invention, automatic annotation generation is performed: the deployed detection model is used to infer the incremental data set at a certain stage, generate pseudo labels, and automatically correct them to achieve automatic generation of new data annotations. Since data cleaning has deleted the data that may be real flying steel, the remaining data are all false detection data, so the flying steel target frame detected is deleted, and only the targets with higher confidence in the remaining target frame are retained to ensure the accuracy of the annotation.

[0114] S640: When the production line stops running, GPU resources are called to perform incremental model training and update based on incremental data sets.

[0115] In one embodiment of the present invention, incremental model learning is performed. When the production line is stopped, the system calls GPU resources to perform incremental model training and updates based on incremental data sets. During the training phase, since the incremental data set may be small, the global update parameters will cause the model, especially the feature extraction part, to be more biased towards new data, and may forget some knowledge in the pre-trained model based on a large amount of labeled data. Therefore, only the detection head module of the yolov8 model is updated to fine-tune the model. Since the scene features change little in the short term and only contain a small amount of false detection data of Feigang, the model can adapt to the new data while maintaining the knowledge learned from the old data. After the model is trained, the previous version of the model is automatically replaced and backed up.

[0116] It should be noted that the incremental update method proposed in the present invention is suitable for when the model enters the stable operation stage and needs to urgently deal with some false alarms, as well as quickly update the model, and perform incremental learning and updating, so that the model can continuously adapt to changes in the scene and maintain a low false detection rate. In addition, the model update method can also use manual global updates. However, the global update requires manual labeling of the flying steel data that occurred within the time period and the false detection data saved by the system, which has a large workload. It is suitable for situations where the model detection rate is low and false detection is serious, such as in the early stage of system operation, or the system runs continuously for a long time, and the effect is significantly deteriorated.

[0117] Second, as Figure 3 As shown, the present invention provides a computer system, which is used to implement the steps of the method described in the first aspect, including a video stream image frame acquisition module, a flying steel target determination module, a dense fog scene flying steel determination module, a comprehensive determination module, a real-time processing module, and a user feedback and incremental learning module;

[0118] The video stream image frame acquisition module is used to obtain real-time video stream image data;

[0119] The flying steel target determination module is used to perform flying steel target detection using a target detection model;

[0120] The dense fog scene flying steel determination module is used to use the image gray value statistical characteristics and classification model to perform comprehensive recognition of flying steel targets in dense fog;

[0121] The comprehensive judgment module is used to make a comprehensive judgment based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog;

[0122] The real-time processing module is used to send an alarm to the server when the comprehensive judgment result shows that flying steel is occurring, start the flying shear device to cut, and record logs and data;

[0123] The user feedback and incremental learning module is used to perform user feedback and model incremental learning.

[0124] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0125] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0126] The present invention provides a flying steel detection method, system, storage medium and program product, which process false detection by introducing scene rule knowledge and flying steel motion pattern experience knowledge, thereby achieving effective control of false judgments; and adding a flying steel recognition module in a dense fog environment to improve the flying steel recognition rate; and then based on the detection data records and user feedback, automatic false detection data screening, labeling and model incremental update are performed, so that the model can adapt to the constantly changing on-site environment, control false detections and improve the robustness of the model.

Claims

1. A flying steel detection method, characterized in that: The following steps are involved: Get real-time video stream image data; Use the target detection model to detect flying steel targets; The image gray value statistical features and classification model are used to comprehensively identify flying steel targets in dense fog; Comprehensive judgment is made based on the flying steel target detection results and the comprehensive identification results of flying steel targets in dense fog; When the comprehensive judgment result shows that flying steel is occurring, an alarm is sent to the server, the flying shear device is turned on to cut, and logs and data are recorded; Conduct user feedback and model incremental learning.

2. The flying steel detection method according to claim 1, characterized in that: The method of using the target detection model to perform Feigang target detection includes: Use the target detection model to infer the image data and obtain the inference results; Based on scene rules and motion patterns, the non-real flying steel targets in the inference results are filtered out to obtain the detection results; Based on the detection results of multiple consecutive frames of images, the flying steel process is determined.

3. The flying steel detection method according to claim 2, characterized in that: The training method of the target detection model includes: Collect Feigang target image data; The flying steel targets are manually labeled as training data to train the target detection model.

4. The flying steel detection method according to claim 2, characterized in that: The filtering of non-real flying steel targets in the inference results based on scene rules and motion patterns includes: Adopting scene rules, based on the flow area of ​​steel and the actual coverage of flying steel, the region of interest is divided and the detection targets outside the region of interest are screened out; Identify the motion pattern of the detection target based on the position and size changes of the detection frame, and screen out the detection targets that do not conform to the Feigang motion pattern.

5. The flying steel detection method according to claim 2, characterized in that: The method for determining the flying steel process is as follows: setting a determination time and a determination number, and when the number of times the flying steel target appears within the determination time is above the determination number, the determination is true, that is, flying steel is occurring on site.

6. The flying steel detection method according to claim 1, characterized in that: The method of using image gray value statistical features and classification models to comprehensively identify flying steel targets in dense fog includes: Calculate the statistical features of the grayscale value of the image, perform statistical analysis based on the feature values ​​of the image of flying steel in dense fog, the image without flying steel in dense fog, and the image in non-dense fog scenes, set the threshold of each feature dimension, and set comprehensive judgment rules to identify the flying steel in dense fog during real-time detection; Build binary classification models for flying steel in dense fog and other scenes, infer the real-time collected images, identify whether they are flying steel in dense fog, and enhance samples through HSV transformation and flying steel halo size, position, and shape transformation to train the binary classification model; A comprehensive judgment is made on whether flying steel occurs in dense fog. When the characteristic value satisfies the comprehensive judgment rules and the classification confidence exceeds the preset threshold, it is judged to be true, that is, flying steel is occurring on site.

7. The flying steel detection method according to claim 1, characterized in that: The comprehensive determination based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog includes: If either the flying steel target detection result or the flying steel target comprehensive identification result under dense fog is judged to be true, it is comprehensively judged that flying steel is occurring at the scene.

8. The flying steel detection method according to claim 1, characterized in that: The user feedback includes: When an alarm record is a real flying steel, the feedback is a real flying steel; When no flying steel error alarm occurs, the feedback is a false alarm; When flying steel occurs on site and there is no alarm record on the client side, the user manually enters the missed alarm information.

9. The flying steel detection method according to claim 1, characterized in that: The model incremental learning includes: Obtain false detection data based on user feedback; Clean the false detection data, missed data, and data within the production line distance threshold to obtain an incremental data set; Use the detection model to infer the incremental data set and generate pseudo labels; When the production line is stopped, GPU resources are called to perform incremental model training and updates based on incremental data sets.

10. A computer system, characterized in that: The computer system is used to implement the steps of the method described in any one of claims 1 to 9, including a video stream image frame acquisition module, a flying steel target determination module, a dense fog scene flying steel determination module, a comprehensive determination module, a real-time processing module, and a user feedback and incremental learning module; The video stream image frame acquisition module is used to obtain real-time video stream image data; The flying steel target determination module is used to perform flying steel target detection using a target detection model; The dense fog scene flying steel determination module is used to use the image gray value statistical characteristics and classification model to perform comprehensive recognition of flying steel targets in dense fog; The comprehensive judgment module is used to make a comprehensive judgment based on the flying steel target detection result and the flying steel target comprehensive identification result under dense fog; The real-time processing module is used to send an alarm to the server when the comprehensive judgment result shows that flying steel is occurring, start the flying shear device to cut, and record logs and data; The user feedback and incremental learning module is used to perform user feedback and model incremental learning.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images