Coal mine environment personnel identification method and system based on algorithm fusion

Abstract

The invention discloses a coal mine environment personnel identification method based on algorithm fusion, and the method comprises the following steps: S1, a data collection module collects a visible light image and an infrared image; s2, extracting features of the visible light image and the infrared image by using an independent feature extraction network to obtain a visible light modal feature FRGB and an infrared modal feature FIR; s3, carrying out weight fusion and channel splicing fusion on the visible light modal feature FRGB and the infrared modal feature FIR to obtain a fusion feature Ffusion; s4, inputting the feature Ffusion into a detection head of the target detection network to obtain a detection result Y; and S5, outputting the detection result Y. According to the coal mine environment personnel identification method and system based on algorithm fusion provided by the invention, through fusion of infrared and visible light image features, the detection precision is improved, and data enhancement is realized in combination with the generative adversarial network so as to adapt to complex scene changes.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence, target detection and multimodal image processing, and in particular to a method and system for identifying personnel in a coal mine environment based on algorithm fusion. Background Art

[0002] In coal mining environments, production operations are often accompanied by large amounts of coal dust. Furthermore, due to insufficient or complex lighting conditions (e.g., alternating strong and weak light) underground, targets in video surveillance footage are often blurred or invisible. This environment poses a significant challenge to traditional personnel detection algorithms based on single-modal images (such as RGB images). Traditional algorithms struggle to accurately detect and identify blurred targets in coal mining environments characterized by complex lighting and high noise levels, leading to missed and false detections. Infrared images, with their strong anti-interference capabilities and night vision characteristics, can effectively complement visible light images. However, leveraging the complementary strengths of visible and infrared images to develop a high-precision, low-latency algorithm that improves the robustness and reliability of personnel recognition has become a key issue that needs to be addressed.

[0003] Application number CN202210967239.0 discloses a personnel detection system and method for dangerous restricted areas for industrial-grade applications, including: obtaining visible light images, infrared images and temperature feedback data of the judgment area; extracting features from the visible light images and infrared images through a deep learning network; processing the visible light images after feature extraction through a visible light personnel recognition algorithm, and importing the visible light personnel recognition results into a visible light personnel recognition module for personnel determination; processing the infrared images after feature extraction through an infrared personnel recognition algorithm, using a temperature module to process the temperature feedback data, and then importing the infrared personnel recognition results and temperature measurement results into the infrared personnel recognition module for personnel determination. Summary of the Invention

[0004] The present invention aims to overcome the technical problem of low accuracy in personnel target detection due to coal dust and lighting problems in the existing technology, and provides a method and system for personnel recognition in a coal mine environment based on algorithm fusion. By fusing infrared and visible light image features, the detection accuracy is improved, and at the same time, data enhancement is achieved by combining generative adversarial networks (GANs) to adapt to complex scene changes.

[0005] The present invention provides a method for identifying personnel in a coal mine environment based on algorithm fusion, comprising the following steps:

[0006] S1. Data acquisition: The data acquisition module collects visible light images and infrared images;

[0007] S2. Feature extraction: Use independent feature extraction networks to extract features from visible light images and infrared images respectively, and obtain visible light modal features F RGBand infrared modal characteristics F IR ;

[0008] F RGB =f RGB (X RGB ,θ RGB )

[0009] F IR =f IR (X 1B ,θ IR )

[0010] Where: f RGB is the feature extraction network of the visible light modality; f IR is the feature extraction network of infrared modality; θ RGB θI is the parameter of the visible light modality feature extraction network; R Extract network parameters for infrared modality features;

[0011] S3, feature fusion: the visible light modal feature F RGB and infrared modal characteristics F IR After weighted fusion and channel splicing fusion, the fusion feature F is obtained fusion ;

[0012] The weighted fusion process is as follows:

[0013] F fusion =ω RGB ·F RGB +ω IR ·F IR

[0014] ω RGB +ω IR =1

[0015] Where: RGB is the weighting coefficient of visible light modal characteristics, ω IR is the weighting coefficient of infrared modal characteristics;

[0016] The channel splicing and fusion process is as follows:

[0017] F fusion =Contact(F RGB , F 1R , axis = -1)

[0018] Among them: Contact represents the splicing operation in the channel dimension;

[0019] S4, detection head processing: fusion feature F fusion Input into the detection head of the target detection network to obtain the detection result Y:

[0020] Y=f det (F fusion θ det )

[0021] Where: f det is the target detection network, θ det To detect the parameters of the network;

[0022] S5. Result output: Output the detection result Y, which includes the category label of the person target, the bounding box position of the person in the image, and the detection confidence score.

[0023] In the method for identifying personnel in a coal mine environment based on algorithm fusion described in the present invention, as a preferred embodiment, data annotation is performed on the visible light image and the infrared image before step S2. The data annotation method comprises: using a semi-automatic annotation tool in combination with an algorithm to assist in generating a detection frame, and combining point annotation with frame annotation to generate annotation data for feature extraction in step S2;

[0024] Box annotations are generated by point annotation and K-nearest neighbor algorithm;

[0025] The format of the labeled data is COCO format, including the location information and category of the person target.

[0026] The method for identifying people in a coal mine environment based on algorithm fusion described in the present invention is preferably configured to perform data enhancement on the visible light image and the infrared image before step S2. The data enhancement method is as follows:

[0027] Generate fake data based on CycleGAN:

[0028] Migrate daytime visible light images to night scenes to expand low-light datasets;

[0029] Generate pseudo-infrared images from visible light images to enhance the target detection model's ability to learn infrared features;

[0030] Convert visible light images into high-quality infrared images through CycleGAN to generate pseudo-bi-photonic data;

[0031] The pseudo-bilight is used for feature extraction in step S2.

[0032] Cycle Generative Adversarial Network (CycleGAN) is a deep learning model for unsupervised image translation between two different domains.

[0033] The method for identifying personnel in a coal mine environment based on algorithm fusion described in the present invention is preferably implemented in a system comprising:

[0034] Data acquisition module: used to collect visible light images and infrared images;

[0035] Data processing module: used for data annotation and data enhancement of visible light images and infrared images;

[0036] Target detection model: used to generate output results Y based on visible light images and infrared images;

[0037] Data output module: used to feed back the output result Y to the user.

[0038] The present invention provides a method for identifying personnel in a coal mine environment based on algorithm fusion. As an optimal method, the target detection models are YOLOv10 and Co-DETR.

[0039] The YOLO (You Only Look Once) model series is a highly efficient object detection model. Its key feature is its ability to simultaneously locate and classify objects in a single forward pass, enabling real-time object detection. From YOLOv1 to YOLOv10, the YOLO model series has undergone multiple iterations and improvements, gradually improving detection accuracy and speed.

[0040] The Co-DETR model structure is an image recognition model based on deep learning, which can simultaneously identify objects and their environment, thereby achieving joint positioning and recognition of multiple objects in the scene.

[0041] The method for identifying personnel in a coal mine environment based on algorithm fusion described in the present invention, as a preferred embodiment, optimizes the target model by:

[0042] Multi-scale detection: For targets at different distances, a multi-scale feature fusion strategy is adopted;

[0043] Slice reasoning: Slice high-resolution images to improve small target detection capabilities;

[0044] Model compression: Use knowledge distillation to compress the model and adapt it to edge devices.

[0045] The method for identifying personnel in a coal mine environment based on algorithm fusion described in the present invention preferably uses a comprehensive loss function, classification loss, and bounding box regression loss for training the objective function:

[0046] L=L cls +λ·L bbox

[0047] Where: L cls is the classification loss, L bbox is the bounding box regression loss, and λ is the balance coefficient.

[0048] The present invention has the following advantages:

[0049] (1) The present invention adopts multimodal feature fusion, which significantly improves the robustness and accuracy of person recognition by combining infrared and visible light features;

[0050] (2) The present invention uses GAN to generate pseudo-bi-optical data, expands the dataset size, and enhances the generalization ability of the model;

[0051] (3) The present invention adopts a semi-supervised annotation strategy, combining the generation methods of point annotation and box annotation, which greatly reduces the cost of manual annotation;

[0052] (4) The present invention adopts adaptive edge deployment, optimizes the algorithm model for the coal mine environment, and supports low-power real-time computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for identifying personnel in coal mine environments based on algorithm fusion. DETAILED DESCRIPTION

[0054] Example 1

[0055] like Figure 1 As shown, a method for identifying personnel in a coal mine environment based on algorithm fusion includes the following steps:

[0056] S1. Data acquisition: The data acquisition module collects visible light images and infrared images;

[0057] S2. Feature extraction: Use independent feature extraction networks to extract features from visible light images and infrared images respectively, and obtain visible light modal features F RGB and infrared modal characteristics F IR ;

[0058] F RGB =f RGB (X RGB ,θ RGB )

[0059] F IR =f IR (X IR ,θ IR )

[0060] Where: f RGB is the feature extraction network of the visible light modality; f IR is the feature extraction network of infrared modality; θ RGB is the parameter of the visible light modality feature extraction network; θ IR Extract network parameters for infrared modality features;

[0061] S3, feature fusion: the visible light modal feature F RGBand infrared modal characteristics F IR After weighted fusion and channel splicing fusion, the fusion feature F is obtained fusion ;

[0062] The weighted fusion process is as follows:

[0063] F fusion =ω RGB ·F RGB +ω IR ·F IR

[0064] ω RGB +ω IR =1

[0065] Where: RGB is the weighting coefficient of visible light modal characteristics, ω IR is the weighting coefficient of infrared modal characteristics;

[0066] The channel splicing and fusion process is as follows:

[0067] F fusion =Contact(F RGB , F 1R , axis = -1)

[0068] Among them: Contact represents the splicing operation in the channel dimension;

[0069] S4, detection head processing: fusion feature F fusion Input into the detection head of the target detection network to obtain the detection result Y:

[0070] Y=f det (F fusion θ det )

[0071] Where: f det is the target detection network, θ det To detect the parameters of the network;

[0072] S5. Result output: Output the detection result Y, which includes the category label of the person target, the bounding box position of the person in the image, and the detection confidence score.

[0073] A system for implementing the method includes:

[0074] Data acquisition module: used to collect visible light images and infrared images;

[0075] Data processing module: used for data annotation and data enhancement of visible light images and infrared images;

[0076] Target detection model: used to generate output results Y based on visible light images and infrared images. The target detection models are YOLOv10 and Co-DETR;

[0077] Data output module: used to feed back the output result Y to the user.

[0078] Optimization of the target model includes:

[0079] Multi-scale detection: For targets at different distances, a multi-scale feature fusion strategy is adopted;

[0080] Slice reasoning: Slice high-resolution images to improve small target detection capabilities;

[0081] Model compression: Use knowledge distillation to compress the model and adapt it to edge devices;

[0082] The training of the objective function uses a comprehensive loss function, classification loss and bounding box regression loss:

[0083] L=L cls +λ·L bbox

[0084] Where: L cls is the classification loss, L bbox is the bounding box regression loss, and λ is the balance coefficient.

[0085] Example 2

[0086] In this embodiment, based on the first embodiment, data annotation is performed on the visible light image and the infrared image before step S2. The data annotation method is: using a semi-automatic annotation tool in combination with an algorithm to assist in generating a detection frame, and combining point annotation with frame annotation to generate annotation data for feature extraction in step S2;

[0087] Box annotations are generated by point annotation and K-nearest neighbor algorithm;

[0088] The format of the labeled data is COCO format, including the location information and category of the person target.

[0089] Example 3

[0090] Based on Example 1, this embodiment performs data enhancement on the visible light image and the infrared image before step S2. The data enhancement method is as follows:

[0091] Generate fake data based on CycleGAN:

[0092] Migrate daytime visible light images to night scenes to expand low-light datasets;

[0093] Generate pseudo-infrared images from visible light images to enhance the target detection model's ability to learn infrared features;

[0094] Convert visible light images into high-quality infrared images through CycleGAN to generate pseudo-bi-photonic data;

[0095] The pseudo-bilight is used for feature extraction in step S2.

[0096] The hardware systems used in Examples 1 to 3 are as follows:

[0097] (1) Hardware Platform

[0098] Equipment: Based on the RK3588 embedded computing platform, integrated with an independent NPU module;

[0099] Performance: Supports 30fps real-time inference, with single-frame processing latency less than 50ms;

[0100] (2) Software Architecture

[0101] Operating system: Support Linux and OpenHarmony;

[0102] Deployment framework: Inference based on ONNX format models, supporting edge device deployment;

[0103] (3) Communication protocol

[0104] Support ONVIF protocol to achieve interconnection with other monitoring devices;

[0105] Supports Modbus TCP protocol for interaction with mine control systems.

[0106] The above description is only illustrative of the present invention and not restrictive. Those skilled in the art will understand that any modification, change or equivalent that can be made without departing from the spirit and scope defined by the claims will fall within the scope of protection of the present invention.

Claims

1. A method for identifying personnel in a coal mine environment based on algorithm fusion, characterized by: The following steps are involved: S1. Data acquisition: The data acquisition module collects visible light images and infrared images; S2. Feature extraction: Extract features from the visible light image and the infrared image using independent feature extraction networks to obtain visible light modal features F RGB and infrared modal characteristics F IR ; F RGB =f RGB (X RGB ,θ RGB ) F IR =f IR (X IB， θ IR ) Where: f RGB is the feature extraction network of the visible light modality; f IR is the feature extraction network of infrared modality; θ RGB θI is the parameter of the visible light modality feature extraction network; R Extract network parameters for infrared modality features; S3, feature fusion: the visible light modal feature F RGB and the infrared modal characteristics F IR After weighted fusion and channel splicing fusion, the fusion feature F is obtained fusion ; The weighted fusion process is as follows: F fusion =ω RGB ·F RGB +oh IR ·F IR oh RGB +oh IR =1 Where: RGB is the weighting coefficient of visible light modal characteristics, ω IR is the weighting coefficient of infrared modal characteristics; The channel splicing and fusion process is as follows: F fusion =Contact(F RGB ,F IB ,axis=-1) Among them: Contact represents the splicing operation in the channel dimension; S4, detection head processing: the fusion feature F fusion Input into the detection head of the target detection network to obtain the detection result Y: Y=f det (F fusion ;θ det ) Where: f det is the target detection network, θ det To detect the parameters of the network; S5. Result output: Output the detection result Y, which includes the category label of the person target, the bounding box position of the person in the image, and the detection confidence score.

2. The method for identifying personnel in a coal mine environment based on algorithm fusion according to claim 1, characterized in that: Before step S2, data annotation is performed on the visible light image and the infrared image. The data annotation method is: using a semi-automatic annotation tool in combination with an algorithm to assist in generating a detection frame, and combining point annotation with frame annotation to generate annotation data for feature extraction in step S2; The box annotation is generated by point annotation and K-nearest neighbor algorithm; The format of the labeled data is COCO format, including the location information and category of the person target.

3. The method for identifying personnel in a coal mine environment based on algorithm fusion according to claim 1, characterized in that: Before step S2, data enhancement is performed on the visible light image and the infrared image. The data enhancement method is: Generate fake data based on CycleGAN: Migrating the visible light images during the day to night scenes to expand the low-light dataset; Generating a pseudo infrared image from the visible light image to enhance the learning ability of the target detection model for infrared features; Converting the visible light image into a high-quality infrared image through CycleGAN to generate pseudo bi-optical data; The pseudo-bifocal light is used for feature extraction in step S2.

4. A method for identifying personnel in a coal mine environment based on algorithm fusion according to any one of claims 1 to 3, characterized in that: A system for implementing the method includes: Data acquisition module: used for acquiring the visible light image and the infrared image; Data processing module: used for performing data annotation and data enhancement on the visible light image and the infrared image; Target detection model: used to generate the output result Y according to the visible light image and the infrared image; Data output module: used to feed back the output result Y to the user.

5. The method for identifying personnel in a coal mine environment based on algorithm fusion according to claim 4, characterized in that: The target detection models are YOLOv10 and Co-DETR.

6. The method for identifying personnel in a coal mine environment based on algorithm fusion according to claim 5, characterized in that: The optimization of the target model includes: Multi-scale detection: For targets at different distances, a multi-scale feature fusion strategy is adopted; Slice reasoning: Slice high-resolution images to improve small target detection capabilities; Model compression: Use knowledge distillation to compress the model and adapt it to edge devices.

7. The method for identifying personnel in a coal mine environment based on algorithm fusion according to claim 5, characterized in that: The training of the objective function uses a comprehensive loss function, classification loss and bounding box regression loss: L=L cls +λ·L bbox Where: L cls is the classification loss, L bbox is the bounding box regression loss, and λ is the balance coefficient.

Images