Industrial instrument panel target detection method and system based on optical flow characteristics and YOLOv8

Abstract

The invention discloses an industrial instrument panel target detection method and system based on optical flow features and YOLOv8, and relates to the technical field of image processing. The method is an industrial dashboard category detection method suitable for a dynamic video scene, the core is to combine optical flow features and a YOLOv8 target detection model, a Transform module is introduced to perform optical flow feature extraction, correlation softmax is utilized to capture inter-frame motion correlation, dynamic features of a target area are highlighted in combination with a self-attention mechanism, and the target area is subjected to target detection. And meanwhile, background interference is suppressed, so that the optical flow features are more accurate and reliable. Through the structure optimization of the YOLOv8 model, the optical flow features are deeply fused into each key module of the detection network, the detection precision, robustness and real-time performance of the instrument panel category in a dynamic video scene are significantly improved, and an efficient and reliable solution is provided for dynamic target identification in a complex industrial environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an industrial instrument panel target detection method and system based on optical flow features and YOLOv8. Background Art

[0002] In a modern industrial environment, industrial instrument panels are key monitoring and control devices. Different types of instrument panels carry important parameters of the operating state of equipment, and their accurate identification is directly related to the safety and efficiency of industrial production. However, industrial scenarios are usually accompanied by situations such as changing lighting, complex backgrounds, and fast-moving targets. Traditional solutions based on manual detection or static image processing often struggle to meet actual requirements. Especially when dealing with the recognition of target categories in dynamic videos, there are obvious limitations in both efficiency and accuracy.

[0003] With the rapid development of deep learning and computer vision technologies, automated instrument panel detection technologies based on target detection models have gradually attracted attention. Especially in dynamic video surveillance tasks, using efficient target detection models and motion information extraction technologies has become an important development direction. In this context, the YOLO target detection model and optical flow feature extraction technology are two very important research directions.

[0004] The YOLO (You Only Look Once) target detection model occupies an important position in real-time target detection tasks with its efficient single-stage detection structure. Through a single forward pass, the YOLO model can simultaneously complete the prediction of the target's position and the recognition of its category. Its excellent speed and accuracy have made it widely used in various scenarios, such as video surveillance, autonomous driving, and industrial inspection. As the latest version of the YOLO series, YOLOv8 further enhances the multi-scale target detection ability by optimizing the Backbone and Neck modules. Especially in static scenarios, its performance in target classification and localization is particularly outstanding.

[0005] However, the original design of the YOLO model mainly targets static image target detection. The detection process relies on the spatial features of a single frame image and lacks the ability to deeply model the temporal information and object motion characteristics between consecutive frames. This limitation in design makes YOLO's detection performance prone to decline in dynamic video scenarios, especially in the industrial instrument panel category detection task, when the instrument panel moves quickly, is occluded, or there are lighting changes. In recent years, some improved versions (such as YOLOv4-CSP and YOLO-TLA) have tried to enhance YOLO's temporal modeling ability through temporal convolution or temporal attention mechanisms, but these methods mostly focus on the fusion of temporal information in simple scenarios, and there is still limited performance improvement when dealing with category recognition tasks in complex dynamic environments.

[0006] Optical flow technology captures the dynamic characteristics of targets by analyzing the pixel motion information between adjacent video frames and is one of the important technologies in the field of video analysis. In tasks such as target tracking, action recognition, and video stabilization, optical flow technology demonstrates excellent capabilities due to its sensitivity to the target's motion trajectory. For example, traditional algorithms like the Lucas-Kanade algorithm and the Horn-Schunck algorithm can estimate the pixel displacements between video frames, providing good solutions for target motion analysis in simple scenarios.

[0007] With the development of deep learning, neural network-based optical flow technologies (such as FlowNet and RAFT) have further improved the accuracy and robustness of optical flow estimation, making them perform more stably when dealing with complex backgrounds or illumination changes. Such technologies can extract deeper motion information from videos, providing rich features for dynamic video target detection. However, the application of optical flow technology in target detection is still in its initial stage and cannot fully exploit the potential of optical flow in spatio-temporal information modeling. Especially in complex dynamic industrial environments, the detection effect for dashboard categories is still insufficient.

[0008] In summary, the YOLO-based target detection technology has significant advantages in real-time performance and accuracy but lacks the utilization of target motion information in dynamic scenarios. Optical flow technology can provide rich motion information for video target detection tasks, but its combination with target detection models mostly remains at a shallow level and fails to show ideal effects in complex dynamic scenarios. In the industrial dashboard category detection task, the limitations of these two technologies are particularly prominent. Summary of the Invention

[0009] Aiming at the deficiencies of the existing technology, the present invention provides an industrial dashboard target detection method and system based on optical flow features and YOLOv8, overcoming the deficiencies of the existing technology in the dynamic detection of industrial dashboard categories, solving the problems of the weak ability of the existing YOLO target detection model to model temporal information and the insufficient depth of the combination of optical flow technology and the detection model, and proposing a solution that can achieve high-precision and real-time detection in complex dynamic video scenarios.

[0010] The first aspect of the present invention provides an industrial dashboard target detection method based on optical flow features and YOLOv8, including the following steps:

[0011] Construct an industrial dashboard target detection dataset;

[0012] Preprocess the samples in the industrial dashboard target detection dataset and divide the preprocessed industrial dashboard target detection dataset into a training set, a validation set, and a test set according to a set ratio;

[0013] Construct an industrial dashboard target detection model;

[0014] Train the industrial instrument panel target detection model using the training set and the validation set to obtain a trained industrial instrument panel target detection model;

[0015] Obtain the video of the industrial instrument panel to be detected and extract two consecutive frames of video images from it, and input them into the trained industrial instrument panel target detection model to obtain the detection result.

[0016] Furthermore, the industrial instrument panel target detection dataset includes a number of samples, and each sample includes two consecutive frames of video images I t-1 、I t extracted from a video containing an industrial instrument panel and the industrial instrument panel category label corresponding to each frame of video image, where I t is the current frame of video image and I t-1 is the adjacent frame of video image.

[0017] The preprocessing is to normalize and adjust the two consecutive frames of video images in each sample to a set size.

[0018] The industrial instrument panel target detection model includes an optical flow feature extraction module, a dimension adjustment module, and a YOLOv8 model;

[0019] The optical flow feature extraction module is used to extract features from the input two consecutive frames of video images to obtain optical flow features as the input of the dimension adjustment module;

[0020] The dimension adjustment module is used to perform dimension conversion on the optical flow features to obtain the optical flow features after dimension conversion to match the input of the YOLOv8 model;

[0021] The YOLOv8 model performs multi-scale feature extraction on the input current frame of video image through the backbone network to obtain spatial features of different scales, uses the optical flow features after dimension conversion as additional inputs to the backbone network, and fuses them with the spatial features of different scales respectively to obtain fused features of different scales; then aggregates the fused features of different scales through the neck network to obtain aggregated feature maps of different scales; finally, based on the aggregated feature maps of different scales, uses the detection head network to achieve target detection to obtain the detection results, including: target bounding boxes, target categories, and motion information; the motion information includes the motion direction and speed of the target; the target is an industrial instrument panel.

[0022] Furthermore, the optical flow feature extraction module includes a convolutional neural network and a Transformer module;

[0023] The convolutional neural network is used to process the input two consecutive frames of video images I t and I t+1Feature extraction is performed to generate the first feature map feature1 and the second feature map feature2 respectively as the inputs of the Transformer module.

[0024] For the Transformer module, the first feature map feature1 and the second feature map feature2 respectively pass through the self-attention mechanism to obtain two intermediate feature maps; then the cross-attention mechanism is used to perform weighted summation on the two intermediate feature maps, and finally, another self-attention mechanism is passed through to finally output the optical flow feature flow_feature.

[0025] Furthermore, the dimension adjustment module is a convolutional network.

[0026] The second aspect of the present invention provides an industrial instrument panel target detection system based on optical flow features and YOLOv8, which is used to implement the industrial instrument panel target detection method based on optical flow features and YOLOv8, including:

[0027] A video acquisition module, which is used to acquire a video containing an industrial instrument panel;

[0028] A video frame acquisition module, which is used to acquire two consecutive video images I t-1 and I t ;

[0029] An industrial instrument panel target detection model, which is used to perform industrial instrument panel detection according to two consecutive video images I t-1 and I t to obtain detection results, including: target bounding boxes, target categories, and motion information.

[0030] The third aspect of the present invention provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 are executed.

[0031] The fourth aspect of the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run by a processor, the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 as described above are executed.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] Existing YOLO models perform excellently in static image detection. However, in dynamic videos, due to the inability to effectively utilize the motion information between frames, their detection capabilities for fast-moving targets or occluded scenarios are limited. In this invention, by deeply integrating optical flow features into the YOLOv8 model, the model can capture the motion characteristics of targets, make up for the information loss in single-frame detection, and thus significantly improve the detection accuracy and robustness of industrial instrument panels.

[0034] Aiming at the challenges of complex dynamic backgrounds, changing lighting environments, and fast-moving targets in industrial instrument panel video detection tasks. This invention combines optical flow features with the feature extraction and fusion capabilities of YOLOv8, enabling the model to better adapt to complex scenarios and accurately identify instrument panel categories.

[0035] Specifically, the innovations of this invention mainly include the following aspects:

[0036] 1) Deep fusion of optical flow features and YOLOv8: In the Backbone module of YOLOv8, optical flow features are fused with multi-scale spatial features (P3, P4, and P5) to enhance the ability to perceive target motion information. In the Neck module, through the feature aggregation mechanism of FPN and PAN, the detection ability of targets at different scales is further strengthened, enabling the model to have a stronger recognition ability for small targets or partially occluded targets;

[0037] 2) Optimized extraction and modeling of optical flow features: This invention introduces a Transformer module for optical flow feature extraction, uses correlation calculation (correlation softmax) to capture the motion correlation between frames, and combines the self-attention mechanism to highlight the dynamic features of the target area while suppressing background interference, making the optical flow features more accurate and reliable. Description of the Drawings

[0038] Figure 1 It is a flowchart of a method for industrial instrument panel target detection based on optical flow features and YOLOv8 in an embodiment of this invention;

[0039] Figure 2 It is a model structure diagram of a method for industrial instrument panel target detection based on optical flow features and YOLOv8 in an embodiment of this invention. Detailed Embodiment

[0040] The present invention proposes an industrial instrument panel category detection method that combines optical flow features with the YOLOv8 model. It is an industrial instrument panel category detection method applicable to dynamic video scenarios. The core lies in combining optical flow features with the YOLOv8 object detection model. By optimizing the structure of the YOLOv8 model, the optical flow features are deeply integrated into each key module of the detection network, significantly improving the detection accuracy, robustness, and real-time performance of instrument panel categories in dynamic video scenarios, and providing an efficient and reliable solution for dynamic target recognition in complex industrial environments.

[0041] This embodiment provides an industrial instrument panel target detection method based on optical flow features and YOLOv8, as Figure 1 shown, including:

[0042] Step 1: Construct an industrial instrument panel target detection dataset;

[0043] The industrial instrument panel target detection dataset includes a number of samples. Each sample includes two consecutive video images I t-1 , I t extracted from a video containing an industrial instrument panel and the corresponding industrial instrument panel category label for each frame of the video image, where I t is the current frame video image and I t-1 is the adjacent frame video image; the industrial instrument panel category labels include circular instrument panel (sf6), square instrument panel (ammeter), circular pressure instrument panel (thermometer), and intelligent electric energy instrument panel (eem);

[0044] Step 2: Preprocess the samples in the industrial instrument panel target detection dataset and divide the preprocessed industrial instrument panel target detection dataset into a training set, a validation set, and a test set according to a set ratio;

[0045] The preprocessing is to normalize and adjust the two consecutive video images in each sample to a set size;

[0046] Step 3: Construct an industrial instrument panel target detection model;

[0047] The overall architecture of the present invention is based on the YOLOv8 object detection model. By introducing optical flow features into its backbone network and neck network, an end-to-end optimized detection network is formed. The optical flow features are used to capture the motion information between video frames. After being fused with the spatial features of YOLOv8, the ability to model the dynamic changes of targets is enhanced;

[0048] As Figure 2 shown, the industrial instrument panel target detection model includes an optical flow feature extraction module, a dimension adjustment module, and a YOLOv8 model;

[0049] The optical flow feature extraction module is used to extract features from two consecutive input video images, and obtain the optical flow features as the input of the dimension adjustment module;

[0050] The optical flow feature extraction module includes a Convolutional Neural Network (CNN Encoder) and a Transformer module;

[0051] The Convolutional Neural Network is used to extract features from two consecutive input video images I t and I t+1 to extract the motion information of each pixel point, including the motion direction, size and speed of the pixel point, and then generate the first feature map feature1 and the second feature map feature2 respectively as the input of the Transformer module;

[0052] The Transformer module passes the first feature map feature1 and the second feature map feature2 through the self-attention mechanism respectively. By comparing the motion information of different regions, it further strengthens the motion information of the target region and suppresses the interference of the background region to obtain two intermediate feature maps; then uses the cross-attention mechanism to perform weighted summation on the two intermediate feature maps, and finally passes through a self-attention mechanism. Finally, the optical flow feature flow_feature is output. This optical flow feature includes the direction of target motion, target displacement information and target motion speed information, providing stronger dynamic perception ability for subsequent target detection tasks.

[0053] Specifically, the self-attention mechanism first calculates the correlation (attention weight) between each pixel point in the input first feature map feature1 and the second feature map feature2: each pixel point calculates their similarity score by comparing with the motion information of other pixel points. This process enables the model to understand how each pixel point is related to other parts of the image in the current frame, especially the relationship between the target region and the background region. After obtaining the similarity scores, the self-attention mechanism assigns a weight to each pixel according to these similarity scores. This means that the target region (such as the dial pointer on the dashboard) usually gets a higher weight, while the background region (such as the static background or irrelevant environmental parts) is given a lower weight. This way of weight assignment makes the dynamic features of the target region more prominent. The self-attention mechanism recomputes the feature representation of each pixel point using weighted averaging. For the target region, the model will pay more attention to the motion information of this region, thus enhancing the target features. For the background region, its motion information will be suppressed to reduce redundant information and improve computational efficiency, while avoiding the influence of irrelevant background interference on the accuracy of target detection.

[0054] The dimension adjustment module is used to perform dimension conversion on the optical flow features to obtain the optical flow features after dimension conversion, so as to match the input of the YOLOv8 model; the dimension adjustment module is a convolutional network;

[0055] The YOLOv8 model extracts multi-scale features from the input current-frame video image through the backbone network, obtaining spatial features of different scales. The optical flow features after dimension conversion are used as additional inputs to the backbone network and are respectively fused with the spatial features of different scales to obtain fused features of different scales. Then, the neck network performs feature aggregation on the fused features of different scales to obtain aggregated feature maps of different scales. Finally, based on the aggregated feature maps of different scales, the detection head network is used to achieve object detection, and the detection results are obtained, including: object bounding boxes, object categories, and motion information; the motion information includes dynamic information such as the motion direction and speed of the object; the object is an industrial instrument panel;

[0056] The backbone network extracts multi-scale spatial features from the input current-frame video image through layer-by-layer convolution and CSP modules (Cross Stage Partial), obtaining three spatial features of different scales: P3: a high-resolution feature map for capturing the detailed information of small objects; P4: a medium-resolution feature map for feature extraction of medium-sized objects; P5: a low-resolution feature map for capturing the overall information of large objects. The optical flow feature flow_feature is adapted through a convolutional network to make its dimension consistent with the three spatial features of different scales, P3, P4, and P5. The optical flow feature is fused with the three spatial features of different scales, P3, P4, and P5, respectively, through the concatenation method. The three fused features contain both static spatial information and dynamic motion information, thus enhancing the model's perception ability for rapidly changing objects;

[0057] The neck network performs multi-scale feature aggregation on the three fused features through FPN (Feature Pyramid Network) and PAN (Path Aggregation Network). FPN is responsible for extracting semantic information from low-resolution features, and PAN enhances the expressive ability of high-resolution features through upsampling, obtaining three feature maps output by the neck network as the input to the detection head network;

[0058] The detection head network generates object bounding boxes and motion information for each object in the feature map output by the neck network through convolutional operations, and then classifies the objects within each object bounding box to output the object category;

[0059] Step 4: Use the training set and the validation set to train the industrial instrument panel target detection model to obtain a trained industrial instrument panel target detection model;

[0060] Step 5: Obtain the video of the industrial instrument panel to be detected and extract two consecutive video images from it and input them into the trained industrial instrument panel target detection model to obtain the detection result.

[0061] In the present invention, by deeply fusing the optical flow feature with the YOLOv8 model, the detection performance of industrial instrument panel categories in dynamic video scenarios is effectively improved. To verify the effect of the present invention, experiments were carried out on the industrial instrument panel target detection data set, and the mAP (Mean Average Precision) indexes of the original YOLOv8 model and the improved model after fusing the optical flow feature were compared respectively. The experimental results are as follows:

[0062] Table 1: Comparison of model effects (mAP) in the embodiments of the present invention

[0063] Category / mAP YOLOv8 Combination of optical flow features and YOLOv8 Circular instrument panel (sf6) 0.87 0.896 Square instrument panel (ammeter) 0.939 0.942 Circular pressure instrument panel (thermometer) 0.962 0.975 Intelligent electric energy instrument panel (eem) 0.883 0.89 all 0.914 0.926

[0064] As can be seen from Table 1, the overall mAP of the original YOLOv8 model in the industrial instrument panel category recognition task is 0.914, while the overall mAP of the improved model combining the optical flow feature in the present invention is increased to 0.926, achieving a 1.2% performance improvement. This result shows that by introducing the optical flow feature, the model's ability to capture the target motion characteristics in dynamic videos is enhanced, thus improving the accuracy of target detection. Specifically, the YOLOv8 model after fusing the optical flow feature has performance improvements in different instrument panel categories. For example: for the sf6 category, the mAP is increased from 0.87 to 0.896, with an increase of 1.5%; for the ammeter category, the mAP is increased from 0.939 to 0.942, with an increase of 0.3%; for the thermometer category, the mAP is increased from 0.962 to 0.975, with an increase of 1.3%; for the eem category, the mAP is increased from 0.883 to 0.89, with an increase of 0.7%.

[0065] These results show that the improved model performs better in the detection tasks of various instrument panels, providing an efficient and reliable solution for industrial video monitoring.

[0066] This embodiment provides an industrial instrument panel target detection system based on optical flow feature and YOLOv8 for implementing the industrial instrument panel target detection method based on optical flow feature and YOLOv8, including:

[0067] A video acquisition module for acquiring a video containing an industrial instrument panel;

[0068] A video frame acquisition module for acquiring two consecutive video images I from a video containing an industrial instrument panel t-1 and I t ;

[0069] An industrial instrument panel target detection model for performing industrial instrument panel detection based on two consecutive video images I t-1 and I t to obtain detection results, including: target bounding boxes, target categories, and motion information.

[0070] This embodiment provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 are executed.

[0071] This embodiment provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 as described above are executed.

Claims

1. An industrial instrument panel target detection method based on optical flow features and YOLOv8, characterized in that, It includes the following steps: Construct an industrial instrument panel target detection dataset; Preprocess the samples in the industrial instrument panel target detection dataset, and divide the preprocessed industrial instrument panel target detection dataset into a training set, a validation set, and a test set according to a set ratio; Construct an industrial instrument panel target detection model; Use the training set and the validation set to train the industrial instrument panel target detection model to obtain a trained industrial instrument panel target detection model; Obtain the video of the industrial instrument panel to be detected and extract two consecutive video images therefrom and input them into the trained industrial instrument panel target detection model to obtain a detection result.

2. The industrial instrument panel target detection method based on optical flow features and YOLOv8 according to claim 1, wherein, The industrial instrument panel target detection dataset includes a number of samples, and each sample includes two consecutive video images I t-1 , I t and the industrial instrument panel category label corresponding to each video image, where I t is the current frame video image, and I t-1 is the adjacent frame video image.

3. The industrial instrument panel target detection method based on optical flow features and YOLOv8 according to claim 1, wherein The preprocessing is to normalize and adjust two consecutive video images in each sample to a set size.

4. The industrial instrument panel target detection method based on optical flow features and YOLOv8 according to claim 1, characterized in that, The industrial instrument panel target detection model includes an optical flow feature extraction module, a dimension adjustment module, and a YOLOv8 model; The optical flow feature extraction module is used to extract features from two consecutive input video images to obtain optical flow features as the input of the dimension adjustment module; The dimension adjustment module is used to perform dimension conversion on the optical flow features to obtain the optical flow features after dimension conversion to match the input of the YOLOv8 model; The YOLOv8 model performs multi-scale feature extraction on the input current-frame video image through a backbone network to obtain spatial features of different scales, takes the optical flow features after dimension conversion as an additional input of the backbone network, and fuses them with the spatial features of different scales respectively to obtain fused features of different scales; Then, perform feature aggregation on the fused features of different scales through a neck network to obtain aggregated feature maps of different scales; Finally, based on the aggregated feature maps of different scales, use a detection head network to implement target detection to obtain a detection result, including: a target bounding box, a target category, and motion information; the motion information includes the motion direction and speed of the target; the target is an industrial instrument panel.

5. The industrial instrument panel target detection method based on optical flow features and YOLOv8 according to claim 4, wherein, The optical flow feature extraction module includes a convolutional neural network and a Transformer module; The convolutional neural network is used to extract features from two consecutive input video images I t and I t+1 to generate a first feature map feature1 and a second feature map feature2 respectively as the inputs of the Transformer module; The Transformer module respectively passes the first feature map feature1 and the second feature map feature2 through a self-attention mechanism to obtain two intermediate feature maps; then uses a cross-attention mechanism to perform weighted summation on the two intermediate feature maps, and finally passes through a self-attention mechanism to finally output the optical flow feature flow_feature.

6. The industrial instrument panel target detection method based on optical flow features and YOLOv8 according to claim 4, characterized in that, The dimension adjustment module is a convolutional network.

7. An industrial instrument panel target detection system based on optical flow features and YOLOv8, which is used to implement the industrial instrument panel target detection method based on optical flow features and YOLOv8 according to any one of claims 1-6, characterized in that, It includes: A video acquisition module for acquiring a video containing an industrial instrument panel; A video frame acquisition module, configured to obtain two consecutive video images I from a video containing an industrial instrument panel t-1 and I t ; Industrial instrument panel target detection model, used to perform industrial instrument panel detection based on two consecutive video images I t-1 and I t to obtain detection results, including: target bounding box, target category, and motion information.

8. An electronic device, characterized in that, It includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 according to any one of claims 1-6 are executed.

9. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, it executes the steps of the industrial instrument panel target detection method based on optical flow features and YOLOv8 according to any one of claims 1-6.

Images