Sampling personnel safety detection method and device of sampling machine, sampling machine and equipment

Abstract

The invention discloses a sampling personnel safety detection method and device of a sampling machine, the sampling machine and equipment, and relates to the technical field of power plant sampling machines, and the method comprises the steps: obtaining a monitoring image in a sampling region; preprocessing the monitoring image to obtain a preprocessed monitoring image; inputting the preprocessed monitoring image into a trained safety detection model, and identifying vehicles and personnel in the sampling area to obtain an identification result; determining distance information between the person and the vehicle according to the identification result; the distance information is compared with a preset distance threshold value, and whether the person is within a safe distance range or not is judged; and if not, triggering an alarm mechanism. According to the invention, potential safety hazards can be accurately and rapidly identified, an early warning signal is timely sent out, the safety of vehicles and personnel is protected, the defects of original safety protection measures are overcome, and the safety and reliability of sampling operation of a power plant are improved.

Description

Technical Field

[0001] This invention belongs to the technical field of power plant sampling machines, and specifically relates to a method, device, sampling machine, and equipment for safety detection of sampling personnel. Background Technology

[0002] Power plant sampling machines are used to detect vehicles entering and exiting the monitored area of ​​a power plant. Operators typically begin work after the vehicles arrive. However, these vehicles are often large trucks, creating blind spots, and personnel moving around them pose significant safety hazards. Current power plant sampling machines only detect vehicles and lack design features for personnel safety, resulting in deficiencies in both vehicle and personnel safety. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a method, device, sampling machine, and equipment for safety detection of sampling personnel. This method can accurately and quickly identify potential safety hazards, issue timely warning signals, safeguard the safety of vehicles and personnel, fill the gaps in existing safety protection measures, and improve the safety and reliability of power plant sampling operations.

[0004] In a first aspect, the present invention provides a method for safety detection of sampling personnel using a sampling machine. The sampling machine includes a camera module and a control module connected to each other. The camera module acquires monitoring images within a sampling area. The sampling area is an image of an area where vehicles and / or personnel pass through. The method is applied to the control module, and the method includes:

[0005] Acquire monitoring images within the sampling area;

[0006] The monitoring image is preprocessed to obtain the preprocessed monitoring image;

[0007] The preprocessed monitoring image is input into the trained safety detection model to identify vehicles and personnel within the sampling area, and the identification results are obtained.

[0008] The distance information between the person and the vehicle is determined based on the recognition result;

[0009] The distance information is compared with a preset distance threshold to determine whether the person is within a safe distance range; if not, an alarm mechanism is triggered.

[0010] In an optional implementation, the security detection model includes a target detection module based on the YOLOv11-S network.

[0011] In an optional implementation, the step of preprocessing the monitoring image to obtain a preprocessed monitoring image includes: performing image enhancement on the monitoring image using one or more of the following: mosaic enhancement, geometric transformation, color transformation, grayscale conversion, and histogram equalization, to obtain the preprocessed monitoring image.

[0012] In an optional implementation, the security detection model includes a 3D convolutional module and an LSTM module;

[0013] The three-dimensional convolution module is set between the backbone network and the feature pyramid of the YOLOv11-S network and is used to extract the spatiotemporal features of multiple consecutive monitoring images.

[0014] The LSTM module is positioned in front of the detection head of the YOLOv11-S network and is used to predict the position of the target based on multiple consecutive frames of monitoring images, thereby obtaining a sequence of target positions for the next three frames.

[0015] In an optional embodiment, the sampling machine further includes an infrared camera connected to the control module; the preprocessing of the monitoring image to obtain the preprocessed monitoring image includes:

[0016] The monitoring image and the infrared thermal imaging image are spatiotemporally aligned using timestamps and a cross-modal Transformer module to obtain an aligned image, which is then input into the security detection model.

[0017] The cross-modal Transformer module includes a feature extraction submodule, an encoding submodule, a decoding submodule, and a residual connection submodule.

[0018] The feature extraction submodule is used to extract features from the monitoring image and the infrared thermal imaging image respectively through a dual-branch convolutional neural network to obtain visible light feature map and infrared feature map;

[0019] The encoding submodule is used to perform position encoding and multi-head self-attention calculation on the visible light feature map to obtain an enhanced visible light feature map;

[0020] The decoding submodule is used to generate a query vector based on the enhanced visible light feature map, and to generate a key vector and a value vector based on the infrared feature map; and to determine the cross attention weights using scaled dot product attention to obtain the aligned infrared feature map.

[0021] The residual connection submodule is used to add the aligned infrared feature map to the infrared feature map element by element to obtain the aligned feature map;

[0022] The security detection model includes a spectral fusion module connected to the backbone submodule in the YOLOv11-S network; the spectral fusion module is used to dynamically allocate fusion weights for the visible light channel and the infrared channel.

[0023] In an optional implementation, the sampling machine further includes a radar module connected to the control module; after inputting the preprocessed monitoring image into the trained safety detection model to identify vehicles and personnel within the sampling area and obtain the identification results, the process further includes:

[0024] Acquire point cloud data from the radar module;

[0025] The spatial position of the recognition result is verified based on the point cloud data to correct the image ranging error and obtain the verified recognition result.

[0026] In an optional embodiment, the sampling machine further includes an illuminance detection module connected to the control module. The preprocessed monitoring image is input into a trained safety detection model to identify vehicles and personnel within the sampling area, obtaining identification results, including:

[0027] Obtain illuminance information;

[0028] Determining the detection mode and adjusting the preset distance threshold based on the illuminance information; specifically including:

[0029] When the illuminance is greater than the second illuminance threshold, a pure visible light detection mode is adopted, and the preset distance threshold is D1;

[0030] When the illuminance is greater than the first illuminance threshold and not greater than the second illuminance threshold, the infrared camera is activated and a dual-spectrum detection mode is adopted. The preset distance threshold is D1×b; where b is a preset multiple threshold, and the first illuminance threshold is less than the second illuminance threshold.

[0031] When the illuminance is not greater than the first illuminance threshold, the radar module is activated and the radar ranging compensation mode is adopted. The preset distance threshold is D1×(1+c×(50-current illuminance)), where c is the illuminance compensation coefficient.

[0032] Secondly, the present invention provides a safety detection device for sampling personnel in a sampling machine, comprising:

[0033] The acquisition module is used to acquire monitoring images within the sampling area;

[0034] The preprocessing module is used to preprocess the monitoring image to obtain the preprocessed monitoring image;

[0035] The target detection module is used to input the preprocessed monitoring image into the trained security detection model, identify vehicles and personnel in the sampling area respectively, and obtain the identification results;

[0036] A distance determination module is used to determine the distance information between the person and the vehicle based on the recognition result;

[0037] The early warning module is used to compare the distance information with a preset distance threshold to determine whether the person is within a safe distance range; if not, an alarm mechanism is triggered.

[0038] Thirdly, the present invention provides a sampling machine, including a control module, and a camera module, an infrared camera, a radar device, and an illuminance detection module respectively connected to the control module; wherein, the control module is used to execute the sampling personnel safety detection method of the sampling machine described in the first aspect.

[0039] Fourthly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the foregoing embodiments.

[0040] In a fourth and fifth aspect, the present invention provides a computer-readable medium having processor-executable non-volatile program code, the program code causing the processor to perform the method described in any of the foregoing embodiments.

[0041] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows: The present invention preprocesses the detection image in the sampling area and inputs it into the trained safety detection model to identify vehicles and personnel in the sampling area respectively. Based on the identification results, it determines whether the distance between personnel and vehicles exceeds the safe distance. If so, an alarm is issued. The present invention can accurately and quickly identify potential safety hazards and issue early warning signals in a timely manner, thus safeguarding the safety of vehicles and personnel, filling the gaps in the original safety protection measures, and improving the safety and reliability of power plant sampling operations. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the safety detection method for sampling personnel using a sampling machine provided in an embodiment of the present invention.

[0043] Figure 2 The on-site monitoring images provided in the embodiments of the present invention;

[0044] Figure 3 This is a schematic diagram of personnel and vehicle target recognition provided in an embodiment of the present invention;

[0045] Figure 4This is another schematic diagram of personnel and vehicle target recognition provided in an embodiment of the present invention;

[0046] Figure 5 A comparison chart of the capabilities of YOLOv11 with other algorithms provided in this embodiment of the invention;

[0047] Figure 6 This is a possible structural diagram of the security detection model provided in an embodiment of the present invention;

[0048] Figure 7 A summary diagram of training data categories and locations provided in embodiments of the present invention;

[0049] Figure 8 Feature diagrams provided for embodiments of the present invention;

[0050] Figure 9 This is a schematic diagram of another possible structure of the security detection model provided in an embodiment of the present invention;

[0051] Figure 10 A schematic diagram illustrating the spatiotemporal alignment principle provided in an embodiment of the present invention;

[0052] Figure 11 This is a schematic diagram of the electrical structure of the sampler provided in an embodiment of the present invention;

[0053] Figure 12 This is a schematic diagram of the system principle of the sampling personnel safety detection device for the sampling machine provided in an embodiment of the present invention;

[0054] Figure 13 A schematic diagram of the system principle of an electronic device provided in an embodiment of the present invention.

[0055] In the diagram: 100 - Acquisition module; 200 - Preprocessing module; 300 - Target detection module; 400 - Distance determination module; 500 - Early warning module; 600 - Sampling machine; 601 - Control module; 602 - Camera module; 603 - Readout detection module; 604 - Infrared camera; 605 - Radar device; 1000 - Electronic equipment; 1001 - Communication interface; 1002 - Processor; 1003 - Memory; 1004 - Bus. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0057] Reference Figure 1 A method for safety detection of sampling personnel using a sampling machine includes the following steps S100-S500.

[0058] Step S100: Acquire monitoring images within the sampling area.

[0059] Specifically, the sampling machine includes a connected camera module and a control module, the camera module acquiring monitoring images of the sampling area; such as Figure 2 As shown, the sampling area is an image of the area where vehicles and / or people pass through, and the image includes images of vehicles and people, such as... Figure 3 and Figure 4 As shown. The sampling machine is installed in the sampling room directly above the sampling area. The camera module of the sampling machine continuously monitors the sampling area below 24 hours a day, capturing information about vehicles and surrounding personnel. The control module includes a Linux server and an NVIDIA 3060 graphics card.

[0060] Step S200: Preprocess the monitoring image to obtain the preprocessed monitoring image.

[0061] The key technical point of this embodiment in employing artificial intelligence object detection algorithms lies in automatically identifying and locating objects in images using deep learning models. It utilizes convolutional neural networks to extract features and combines regression and classification techniques to achieve high-precision and high-efficiency object detection. The PyTorch deep learning framework is used, employing the state-of-the-art object detection algorithm YOLOv11 for object detection. Figure 5 As shown in Table 1, YOLOv11 is more accurate and faster than other object detection algorithms.

[0062] Table 1 Comparison of YOLOv11 capabilities with other algorithms

[0063]

[0064]

[0065] Before performing target detection on the monitored images, preprocessing is required, including image enhancement. This embodiment uses image enhancement techniques to improve the model's generalization ability and reduce overfitting. Image enhancement includes one or more of the following: mosaic, mirroring, grayscale adjustment, HSV color transformation, and histogram equalization. These data enhancement techniques improve the model's adaptability to complex scenes.

[0066] Since people have small pixel counts in images, this embodiment scales the image to 1280×1280 resolution as the original input size for the model, preserving the original aspect ratio. This makes the model more accurate in extracting subtle features.

[0067] Step S300: Input the preprocessed monitoring image into the trained security detection model to identify vehicles and personnel within the sampling area, and obtain the identification results, such as... Figure 3 and Figure 4 The detection box shown.

[0068] The security detection model in this embodiment is a neural network based on the YOLOv11-S object detection algorithm. YOLOv11-S has high accuracy and real-time performance and can effectively process video stream data. The specific structure of YOLOv11-S is as follows: Figure 6 As shown, YOLOv11-S comprises a backbone, a neck, and a head. The backbone extracts basic image features layer by layer using Conv (convolutional layers) and C3K2 (efficient cross-stage residual module) to generate multi-scale feature maps (such as edge and texture information at different resolutions). The neck integrates multi-scale features using upsampling (enlarging feature maps), Concat (cross-level feature concatenation), C2PSA (channel-spatial attention fusion), and SPFF (spatial pyramid feature fusion) to enhance the correlation between target localization and semantic information. The head further optimizes features through Conv convolutional layers, and the Detect (detection head) outputs the target location (boundary box) and class prediction to complete the final detection task. This embodiment uses the YOLOv11-S target detection algorithm to detect vehicles and people, and combines distance thresholds to determine whether people are in a safe location during feature extraction. This embodiment uses 32x downsampling and feature fusion to capture local features such as edges, textures, and shapes in the image, thereby improving detection accuracy.

[0069] Step S400: Determine the distance information between the person and the vehicle based on the recognition result.

[0070] For the detected personnel and carriage boundary boxes, the pixel distance is calculated using the following formula.

[0071]

[0072] In the above formula, d represents the distance between people and vehicles, and x person and y person Let x represent the x-coordinate and y-coordinate of the person, respectively. car and y car These represent the horizontal and vertical coordinates of the vehicle, respectively.

[0073] Step S500: Compare the distance information with a preset distance threshold to determine whether the person is within a safe distance range; if not, trigger an alarm mechanism.

[0074] The system determines whether d is less than a preset distance threshold (e.g., 500 pixels). If so, an alarm mechanism is triggered. The alarm mechanism can trigger an audible and visual alarm, as well as send alarm information to the sampling machine's host computer remote terminal (which communicates with the control module).

[0075] The training process of the safety detection model in this embodiment is as follows: A sampling machine is used to collect the video stream from the top of the sampled area, and key frames in which vehicles and personnel are present at the same time are extracted as training data. The data required for target detection (video taken by the camera module on the top of the power plant sampling machine) is labeled, including images and corresponding labels (the category and location of the target object). That is, the label data is obtained by labeling the target category and location (boundary box coordinates) in each image. The target categories include personnel and carriages.

[0076] The training data undergoes mosaic enhancement, geometric transformation, color transformation, grayscale conversion, and histogram equalization. Geometric transformations include rotation, scaling, and cropping. The S-model (Small model) from YOLOv11 is employed, which balances computational efficiency and inspection accuracy, and adapts to high-resolution input. The security detection model is trained using augmented and scaled image and label data, including forward propagation, loss calculation, backpropagation, and parameter updates. During training, hyperparameters such as learning rate and batch size need to be adjusted to optimize model performance. Focal Loss is used to calculate the classification loss to address class imbalance, CIoU Loss (Complete Intersection over Union) is used to calculate the regression loss to optimize bounding box position accuracy, and Binary Cross-Entropy (BCE) is used to determine target presence. In backpropagation, the AdamW optimizer (β1 = 0.9, β2 = 0.999) is used for optimization, with dynamic adjustment of the learning rate (initial value 1e-3, cosine annealing strategy), weight decay of 1e-4, and gradient clipping max_norm = 10 to prevent overfitting. The output includes bounding box coordinates, class, and confidence score.

[0077] In some possible implementations, for the YOLOv11 model, the parameters could be such that, for the downsampling submodule, a 32x downsampling (i.e., from 1280×1280 to 40×40) is achieved through convolutional layers with a stride of 2 to preserve more features of small targets. The activation function is SiLU to balance non-linear expressiveness and computational efficiency. The final output is feature maps at three scales (40×40, 20×20, 10×10), as shown below. Figure 8 As shown.

[0078] Furthermore, the security detection model includes a 3D convolutional module and an LSTM module;

[0079] The three-dimensional convolution module is set between the backbone network and the feature pyramid of the YOLOv11-S network to extract the spatiotemporal features of multiple consecutive monitoring images by combining the following formula (1);

[0080] Δp n =f(v avg )·W n ·Xt-2：t+1 (1)

[0081] In equation (1), Δp n For dynamic offset, v avg W represents the average velocity of the target being measured. n For learnable weights, X t-2：t+1 denoted as feature maps of three adjacent frames; where n is a positive integer.

[0082] The LSTM module is placed before the detector head of the YOLOv11-S network, between the feature pyramid SPFF and C2SPA, as follows. Figure 9 As shown, the target position sequence for the next three frames is obtained by predicting the target position traj_pred based on continuous multi-frame monitoring images and the following formula (2); the displacement error L between adjacent frames is determined based on the following formula (3). speed The fusion loss L is determined based on equations (4) and (5). total ;

[0083] traj_pred={p t+1 p t+2 p t+3},(2)

[0084]

[0085] L traj =1-IoU(traj_pred,traj_gt), (4)

[0086] L total =W1L speed +W2L traj (5)

[0087] In the formula, p t+1 This indicates the position of the target being measured in the next frame; traj_pred represents the predicted trajectory, and traj_gt represents the true trajectory; v t p represents the predicted velocity. t This indicates the position of the target being measured in the current frame, where T is the length of the time window; W1 and W2 are weighting coefficients.

[0088] like Figure 3 and Figure 4 As shown, people are relatively small in the image, and sometimes people and vehicles are in motion. Conventional YOLOv11-S-based security detection models are inaccurate in low-light (overcast or dusky) and moving scenes, relying solely on single-frame monitoring images. Therefore, this embodiment... Figure 6Based on this, the YOLOv11-S network model was improved by inserting a 3D convolutional layer between the last C3K2 layer and the Spatial Pyramid (SPFF) in the YOLOv11 backbone network. This layer is used to extract spatiotemporal features (such as personnel movement trajectories) between consecutive frames, enhancing the ability to capture dynamic targets in low-light conditions. An LSTM module (between SPFF and Detect) is embedded before the detection head to predict the position of personnel by analyzing multiple consecutive monitoring images. The predicted position is then used to determine the distance information between personnel and vehicles. The LSTM module models the temporal correlation of target behavior, solving the misjudgment problem of traditional single-frame detection.

[0089] The 3D convolutional module is placed after the last C3K2 in the backbone network, such as... Figure 9 As shown. The three-dimensional convolution module uses a three-dimensional convolution kernel, which refers to a 3×3×3 spatiotemporal convolution kernel, that is, three consecutive frames of monitoring images in the time dimension, such as: frame t-1, frame t, and frame t+1; and a 3×3 pixel area in the spatial dimension. The three-dimensional convolution kernel can simultaneously capture the changes of the target in time and space, such as the trajectory of personnel movement and the trajectory of vehicle movement. Equation (1) is based on the average velocity v of the target. avg The obtained dynamic offset Δp n This embodiment combines dynamic offset with the convolution kernel, adjusting the sampling position of the convolution kernel according to the target velocity without changing the kernel weights, only shifting the feature map index, that is, shifting the sampling center of the convolution kernel from (x, y) to (x + Δp) on the feature map. x ,y+Δp y Thus, for fast-moving targets, this embodiment can automatically expand the receptive field of the convolution kernel, and for slow-moving targets, it can also maintain the capture of local details.

[0090] The LSTM module (time step ≥ 5) consists of an input layer, a hidden layer, and a fully connected layer. The input layer receives feature vectors from the upper layer, the hidden layer uses bidirectional LSTM units, and the fully connected layer maps the LSTM output to the target position coordinates. By using the LSTM module to perform correlation matching on the target positions in consecutive frames, the temporal continuity of the target motion trajectory is determined, eliminating false alarms caused by instantaneous occlusion and avoiding misjudgments.

[0091] In practice, the alarm mechanism can also be a continuous frame alarm, that is, when a preset distance threshold is reached after a preset number of consecutive frames (e.g., 5 frames), an alarm is triggered to reduce false alarms.

[0092] This embodiment combines a 3D convolution module and an LSTM module. The 3D convolution module can extract local spatiotemporal features and capture the movement direction of people in three consecutive frames of surveillance images, while the LSTM module can model long-term temporal dependencies and capture the complete trajectory of people entering and leaving the monitored area. Compared with using the 3D convolution module or the LSTM module alone, combining the two can capture short-term motion patterns and correlate long-term behaviors, thus improving the accuracy of target detection.

[0093] The sampling machine monitors the monitored area 24 hours a day. In low-light environments such as evening or night, the accuracy of target detection by the sampling machine is easily affected by light. Although power plants usually have sufficient power supply, installing high-brightness streetlights in the monitored area can supplement the lighting and ensure the imaging quality of the camera to a certain extent. However, strong light may cause reflections and glare interference, and streetlights cannot cover the entire monitored area, and the lamps also have a certain probability of malfunction.

[0094] Therefore, in some possible embodiments, the sampling machine also includes an infrared camera connected to the control module; in specific implementations, the deployed camera module and the infrared camera ensure frame synchronization between the two cameras through hardware trigger signals (such as GPIO), with a time error of less than 10 milliseconds. Step S200 also includes the following steps:

[0095] The monitoring image and the infrared thermal imaging image are spatiotemporally aligned using timestamps and cross-modal Transformer modules to obtain an aligned image. This aligned image is then input into the security detection model, which includes a spectral fusion module connected to the backbone submodule in the YOLOv11-S network.

[0096] The sampling machine in this embodiment also includes an infrared camera, which can acquire infrared thermal imaging images. When the infrared camera needs to work simultaneously with the original camera module, the infrared thermal imaging image and the monitoring image (visible light image) are spatiotemporally aligned and then input into the SPFF module in the security detection model. The spectral fusion module is embedded into the SPFF module based on the above equations (6) and (7). The reason for inputting from the SPFF module is that the YOLOv11 backbone network focuses on single-modal feature extraction. Therefore, for the visible light image captured by the camera component, i.e., the monitoring image, it can be input from the first convolutional layer of the backbone network. The feature map after spatiotemporal alignment already has cross-modal information and needs to be directly integrated using the neck network. Since the SPFF module is a spatial pyramid feature fusion module, it is good at performing any multi-scale feature fusion on multi-modal features and is very suitable for processing aligned cross-modal data. It should be noted that the monitoring image and the infrared thermal imaging image before alignment need to maintain the same resolution.

[0097] Figure 10 To achieve spatiotemporal alignment of surveillance images and infrared thermal imaging images, time alignment between the two modalities is first achieved by adding timestamps. Then, a cross-modal Transformer module is used to eliminate resolution and viewpoint differences. The cross-modal Transformer module includes a feature extraction submodule, an encoding submodule, a decoding submodule, and a residual connection submodule.

[0098] The feature extraction submodule is used to extract features from surveillance images and infrared thermal imaging images using a dual-branch convolutional neural network, resulting in visible light feature maps and infrared feature maps. Here, the dual-branch convolutional neural network in the feature extraction submodule consists of two branches: the first branch uses a ResNet architecture to extract visible light image features, and the second branch, composed of three convolutional layers, is used to extract infrared thermal imaging features. The surveillance image is input to the first branch, and the infrared thermal imaging image is input to the second branch for feature extraction.

[0099] The encoding submodule performs position encoding and multi-head self-attention calculation on the visible light feature map to obtain an enhanced visible light feature map. The decoding submodule generates a query vector based on the enhanced visible light feature map, and generates key and value vectors based on the infrared feature map; it also uses scaled dot product attention to determine cross-attention weights to obtain an aligned infrared feature map.

[0100] Specifically, positional encoding is used to determine the positional relationships of each region in each feature map. It divides the image into a grid and labels the row and column numbers. Then, it calculates a sine wave and a cosine wave for the x and y coordinates of each position to obtain a positional fingerprint. This fingerprint is then superimposed onto the image features to determine the position of each grid cell. A multi-head self-attention mechanism is used to understand the correlations between different regions in the feature map. It converts the input features into query vectors, key vectors, and value vectors. The query vector is then matched sequentially with all key vectors to obtain a correlation score. This process is replicated multiple times, with each head focusing on different correlation patterns. Finally, the outputs of all heads are merged to obtain a comprehensively understood feature.

[0101] The cross-attention mechanism enables the computer to accurately search for corresponding information in the "answer library" (key and value vectors) of infrared features, based on the "question" (query vector) posed by visible light features. Specifically, the computer first converts the enhanced visible light features (containing location and global correlation information) into a query vector, representing the semantic clues to be aligned (e.g., "which heat source should the human silhouette match"); simultaneously, it converts the infrared features into key vectors (for similarity matching) and value vectors (carrying the actual feature content). Next, by scaling the dot product, the matching degree between the query vector and the key vector is compared one by one, generating attention weights from 0 to 1 (e.g., 0.9 for people, 0.1 for background). Finally, these weights are used to weight and fuse the infrared feature value vectors, enhancing features in high-weight regions (e.g., human heat sources) and suppressing features in low-weight regions (e.g., interference noise). The final output is an infrared feature map aligned with the visible light space, ensuring that subsequent detection modules can simultaneously utilize the key information from both modalities.

[0102] The residual connection submodule is used to add the aligned infrared feature map element-wise to the original infrared feature map to obtain the aligned feature map. Specifically, the aligned infrared features are directly superimposed with the original infrared features to retain effective information in the original data and optimize feature representation. In detail, the computer adds the aligned infrared feature map (containing cross-modal corrected spatial location information) to the unprocessed original infrared feature map pixel-wise, so that details of the original heat source that may have been lost during alignment (such as weak temperature gradients and edge noise) are preserved, while enhancing key aligned features (such as the precise contour of human body heat sources). This "original + corrected" dual-path fusion mechanism avoids the loss of details caused by over-correction of feature alignment (such as the false elimination of blurred heat sources in smog) and accelerates model convergence through residual learning.

[0103] Furthermore, the spectral fusion module is embedded in the feature pyramid (SPFF) to dynamically allocate the fusion weights α of the visible light channel and the infrared channel according to the following equations (6) and (7);

[0104] α=σ(Conv 1×1 ([F vis ;F ir (6)

[0105] F_fused=α⊙F vis +(1-α)⊙F ir (7)

[0106] In equations (6) and (7), σ is the Sigmoid function, ⊙ is used for element-wise multiplication, and F vis For visible light characteristic maps, F ir This is an infrared light feature map. Conv 1×1It is a convolutional layer with 1 channel.

[0107] Specifically, in low-light scenarios, α approaches 0 and F_fused depends on infrared features, while under normal daylight, especially strong light, α approaches 1 and F_fused preferentially depends on visible light features.

[0108] During training, the convolutional kernels of the cross-modal attention layer are initialized with a Xavier normal distribution, and the learning rate is set to 1.5, higher than other layers. Training can be divided into two stages. In the pre-training stage, the YOLOv11-s model is trained using only visible light data with a learning rate of 1e-3, and the first 3 layers of the backbone are frozen. In the fine-tuning stage, the pre-trained weights are loaded, all layers are unfrozen, and training is performed using bimodal data. The learning rate is reduced to 5e-4, and the loss function weights are adjusted to: classification loss 0.8 and regression loss 1.2 (to enhance localization accuracy).

[0109] When the illumination is very low, the recognition result will have a large error, and it is necessary to compensate for the distance information between people and vehicles. Therefore, in some possible embodiments, the sampling machine also includes a radar module connected to the control module; after step S300, the following steps (1)-(2) are also included.

[0110] (1) Acquire point cloud data from the radar module;

[0111] (2) Based on the point cloud data, the spatial position of the recognition result is verified to correct the image ranging error and obtain the verified recognition result.

[0112] Specifically, the point cloud data is converted from the polar coordinate system to the camera coordinate system, i.e., a three-dimensional coordinate system. For example, x = rcosθ. Where r is the radar ranging value, and θ and These are the pitch angle and azimuth angle, respectively. The following formula is used to calculate the three-dimensional coordinates (X, Y, Z) of the center point of the detection bounding box (person or vehicle detection box) after target detection in the surveillance image. image Y image Z image In other words, mapping the center point of a two-dimensional bounding box to three-dimensional space:

[0113] Z = D(u, v);

[0114] (X, Y, Z) = K -1 ·[u, v, 1] T ·Z;

[0115] In the formula, (u, v) is the center point of the detection box, and K is the camera intrinsic parameter matrix.

[0116] Search for the nearest neighbor in the radar point cloud using K-nearest neighbor (KNN) or radius search, calculate the Euclidean distance, and determine the spatial offset Δd.

[0117]

[0118] The radar power supply provides three-dimensional coordinates (X). radar Y radar Z radar Calculate the radar's three-dimensional coordinates and visual coordinates (X). image Y image Z image The Euclidean distance between points is calculated, resulting in d. Only points within a distance threshold (e.g., d < 1m) are retained, excluding noise points. Then, the spatial offset Δd is calculated based on the data on the X and Y axes (ignoring changes in height Z). When Δd is greater than a preset offset threshold (e.g., 0.5 meters), it is determined to be an image detection error, and the position is corrected using radar data.

[0119] After determining that radar data will be used for correction, the horizontal distance between the vehicle and the person also needs to be calculated using radar data to correct the position. For example, if the identification result is 2.8 meters (distance between the person and the vehicle), while the radar module's measurement result is 3.5 meters (distance between the person and the vehicle), the spatial offset is 0.7 meters. Since the spatial offset is greater than the preset offset threshold of 0.5 meters, the radar measurement result of 3.5 meters is adopted as the final identification result.

[0120] In some possible embodiments, the sampler further includes an illuminance detection module connected to the control module, and step S300 further includes the following steps S310-S320.

[0121] Step S310: Obtain illuminance information.

[0122] Step S320: Determine the detection mode based on the illuminance information and adjust the preset distance threshold; specifically including:

[0123] Step S321: When the illuminance is greater than the second illuminance threshold, the pure visible light detection mode is adopted, and the preset distance threshold is D1.

[0124] Step S322: When the illuminance is greater than the first illuminance threshold and not greater than the second illuminance threshold, the infrared camera is activated and a dual-spectrum detection mode is adopted. The preset distance threshold is D1×b; where b is a preset multiple threshold and the first illuminance threshold is less than the second illuminance threshold.

[0125] Step S323: When the illuminance is not greater than the first illuminance threshold, start the radar module and adopt the radar ranging compensation mode. The preset distance threshold is D1×(1+c×(50-current illuminance)), where c is the illuminance compensation coefficient.

[0126] This embodiment uses a first illuminance threshold of 50 lux, a second illuminance threshold of 200 lux, a preset distance threshold D1 of 4 meters, b of 1.2, and c of 0.005 as an example. When the current illuminance is greater than 200 lux, the preset distance threshold is 4 meters, and a pure visible light detection mode is used. When 50 lux < current illuminance ≤ 200 lux, the preset distance threshold is 4 × 1.2 = 4.8 meters, and a dual-spectrum detection mode is used. When the current illuminance is ≤ 50 lux, for example, when the current illuminance = 20 lux, the preset distance threshold is 4 × (1 + 0.005 × (50 - 20)) = 4.6 meters. If the recognition result (including the recognition result obtained after modification using a radar device) is greater than 4.6 meters, an early warning is issued; if it is less than 4.6 meters, no early warning is issued.

[0127] In some extreme weather conditions, such as fog, dust storms, rain, snow, and strong winds, the clarity of the surveillance images captured by the camera components can be affected, resulting in larger recognition errors. Therefore, in some possible embodiments, step S400 is followed by the following steps (1)-(3).

[0128] Step (1): Obtain the current environmental parameters through the environmental sensor group. The current environmental parameters include dust concentration, humidity and vibration amplitude.

[0129] Step (2): Calculate the risk coefficient result based on the current environmental parameters and the pre-established environmental risk coefficient calculation model.

[0130] For example, the environmental risk coefficient calculation model established for monitoring sandstorms, rain, snow, and strong winds can be: K=α×C+β×H+γ×A, where C is the dust concentration, H is the humidity, A is the vibration amplitude, α, β, and γ are weighting factors, and K is the risk coefficient.

[0131] Step (3) compares the risk coefficient result with the preset environmental risk threshold. When the environmental risk coefficient exceeds the preset environmental risk threshold, the safety threshold range is automatically narrowed and the safety response level is improved.

[0132] The safety threshold range here can be set by referring to the setting principle of the preset distance threshold in steps S322-S323 above, that is, by setting a reasonable safety threshold range through multiple thresholds, compensation coefficients, etc., which will not be elaborated here.

[0133] In some possible embodiments, the method further includes the following steps (1)-(3).

[0134] Step (1): Set up a laser projection device at the boundary of the sampling area to project a dynamic safety warning line in real time.

[0135] Step (2): When a person is detected approaching the warning line, the color of the projection line is controlled to gradually change from green to red, and a pulsed light band is generated.

[0136] Step (3) involves collecting foot contact signals through a pressure-sensitive floor mat and then using a UWB positioning tag to perform dual verification of the person's location.

[0137] In some possible embodiments, the method further includes the following steps (1)-(3).

[0138] Step (1): Establish a historical accident database and record the environmental parameters (illuminance, dust concentration, temperature and humidity, etc.) and detection mode parameters (pure visible light mode, dual-spectrum fusion mode, radar compensation mode) for each safety event.

[0139] Step (2) involves determining the risk response pattern using environmental parameters and detection mode parameters. This can be achieved through analysis of illuminance and detection mode effectiveness, environmental parameters and threshold compensation coefficients, and risk hotspot maps using a random forest algorithm.

[0140] Step (3) involves switching the detection mode and / or dynamically adjusting the threshold based on the current illuminance information and the matching results of the risk hotspots.

[0141] See Figure 11 The present invention provides a sampling machine 600, which includes a control module 601, and a camera module 602, an infrared camera 604, a radar device 605 and an illuminance detection module 603 respectively connected to the control module 601; wherein, the control module 601 is used to execute the aforementioned sampling personnel safety detection method.

[0142] See Figure 12 This invention provides a sampling personnel safety detection device, comprising an acquisition module 100, a preprocessing module 200, a target detection module 300, a distance determination module 400, and an early warning module 500. The acquisition module 100 acquires monitoring images within the sampling area; the preprocessing module 200 preprocesses the monitoring images to obtain preprocessed monitoring images; the target detection module 300 inputs the preprocessed monitoring images into a trained safety detection model to identify vehicles and personnel within the sampling area, obtaining identification results; the distance determination module 400 determines the distance information between personnel and vehicles based on the identification results; and the early warning module 500 compares the distance information with a preset distance threshold to determine whether the personnel are within a safe distance range; if not, an alarm mechanism is triggered.

[0143] In an optional embodiment, the security detection model includes a target detection module based on the YOLOv11-S network.

[0144] In an optional embodiment, the preprocessing module 200 includes performing image enhancement on the monitoring image using one or more of mosaic enhancement, geometric transformation, color transformation, grayscale conversion, and histogram equalization to obtain a preprocessed monitoring image.

[0145] In an optional embodiment, the security detection model includes a 3D convolutional module and an LSTM module;

[0146] The three-dimensional convolution module is set between the backbone network and the feature pyramid of the YOLOv11-S network to extract the spatiotemporal features of multiple consecutive monitoring images in combination with the following formula (1).

[0147] Δp n =f(v avg )·W n ·X t-2：t+1 (1)

[0148] In equation (1), Δp n For dynamic offset, v avg W represents the average velocity of the target being measured. n For learnable weights, X t-2：t+1 denoted as feature maps of three adjacent frames; where n is a positive integer.

[0149] The LSTM module is placed in front of the detection head of the YOLOv11-S network to predict the target position traj_pred based on multiple consecutive monitoring images and the following formula (2), thereby obtaining the target position sequence for the next three frames; wherein, during the training of the security detection model, the displacement error L between adjacent frames is determined based on the following formula (3). speed The fusion loss L is determined based on equations (4) and (5). total ;

[0150] traj_pred={p t+1 p t+2 p t+3},(2)

[0151]

[0152] L traj =1-IoU(traj_pred,traj_gt), (4)

[0153] L total =W1L speed +W2L traj (5)

[0154] In the formula, p t+1This indicates the position of the target being measured in the next frame; traj_pred represents the predicted trajectory, and traj_gt represents the true trajectory; v t p represents the predicted velocity. t This indicates the position of the target being measured in the current frame, where T is the length of the time window; W1 and W2 are weighting coefficients.

[0155] In an optional embodiment, the sampling machine further includes an infrared camera connected to the control module; the preprocessing module 200 includes a spatiotemporal alignment module. The spatiotemporal alignment module is used to perform spatiotemporal alignment of the monitoring image and the infrared thermal imaging image using timestamps and the cross-modal Transformer module to obtain an aligned image, which is then input into the security detection model.

[0156] The cross-modal Transformer module includes a feature extraction submodule, an encoding submodule, a decoding submodule, and a residual connection submodule. The feature extraction submodule extracts features from the surveillance image and the infrared thermal imaging image using a dual-branch convolutional neural network, obtaining visible light feature maps and infrared feature maps respectively. The encoding submodule performs position encoding and multi-head self-attention calculation on the visible light feature map to obtain an enhanced visible light feature map. The decoding submodule generates a query vector based on the enhanced visible light feature map and a key vector and value vector based on the infrared feature map; it also uses scaled dot product attention to determine the cross-attention weights, resulting in an aligned infrared feature map. The residual connection submodule adds the aligned infrared feature map element-wise to the original infrared feature map to obtain the final aligned feature map.

[0157] The security detection model includes a spectral fusion module connected to the backbone sub-module in the YOLOv11-S network; the spectral fusion module is used to dynamically allocate the fusion weights α of the visible light channel and the infrared channel according to the following equations (6) and (7).

[0158] α=σ(Conv 1×1 ([F vis ;F ir (6)

[0159] F_fused=α⊙F vis +(1-α)⊙F ir (7)

[0160] In equations (6) and (7), σ is the Sigmoid function, ⊙ is used for element-wise multiplication, and F vis For visible light characteristic maps, F ir This is an infrared light feature map.

[0161] In an optional embodiment, the sampling machine further includes a radar module connected to the control module; after the target detection module 30, it also includes a radar detection module and a verification module. The radar detection module is used to acquire point cloud data from the radar module. The verification module is used to perform spatial position verification on the recognition result based on the point cloud data to correct image ranging errors and obtain a verified recognition result.

[0162] In an optional embodiment, the sampling machine further includes an illuminance detection module connected to the control module. The target detection module 300 includes an illuminance module and an adjustment module. The illuminance module is used to acquire illuminance information. The adjustment module is used to determine the detection mode and adjust a preset distance threshold based on the illuminance information. The adjustment module includes a visible light sub-module, a dual-spectrum detection sub-module, and a radar ranging compensation sub-module. The visible light detection sub-module is used to adopt a pure visible light detection mode when the illuminance is greater than a second illuminance threshold, with a preset distance threshold of D1. The dual-spectrum detection sub-module is used to activate the infrared camera and adopt a dual-spectrum detection mode when the illuminance is greater than a first illuminance threshold but not greater than a second illuminance threshold, with a preset distance threshold of D1×b; where b is a preset multiple threshold, and the first illuminance threshold is less than the second illuminance threshold. The radar ranging compensation sub-module is used to activate the radar module and adopt a radar ranging compensation mode when the illuminance is not greater than the first illuminance threshold, with a preset distance threshold of D1×(1+c×(50-current illuminance)), where c is the illuminance compensation coefficient.

[0163] The apparatus provided in the embodiments of this application has the same inventive concept as the method provided in the embodiments of this application. As long as the method can solve the technical problem, the apparatus can also solve the technical problem. This will not be elaborated here.

[0164] Reference Figure 13 The present invention also provides an electronic device 1000, including a communication interface 1001, a processor 1002, a memory 1003, and a bus 1004. The processor 1002, the communication interface 1001, and the memory 1003 are connected via the bus 1004. The memory 1003 is used to store a computer program that supports the processor 1002 in executing the above-described method. The processor 1002 is configured to execute the program stored in the memory 1003.

[0165] Optionally, embodiments of the present invention also provide a computer-readable medium having non-volatile program code executable by a processor 1002, the program code causing the processor 1002 to perform the methods as described in the above embodiments.

[0166] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A method for safety detection of sampling personnel using a sampling machine, characterized in that, The sampling device includes a camera module and a control module connected to each other. The camera module acquires monitoring images within a sampling area. The sampling area is an area through which vehicles and / or people pass. The method is applied to the control module, and the method includes: Acquire monitoring images within the sampling area; The monitoring image is preprocessed to obtain the preprocessed monitoring image; The preprocessed monitoring image is input into the trained safety detection model to identify vehicles and personnel within the sampling area, and the identification results are obtained. The distance information between the person and the vehicle is determined based on the recognition result; The distance information is compared with a preset distance threshold to determine whether the person is within a safe distance range; if not, an alarm mechanism is triggered.

2. The method according to claim 1, characterized in that, The security detection model includes a target detection module based on the YOLOv11-S network.

3. The method according to claim 1, characterized in that, The step of preprocessing the monitoring image to obtain a preprocessed monitoring image includes: The monitoring image is enhanced by one or more of the following methods: mosaic enhancement, geometric transformation, color transformation, grayscale conversion, and histogram equalization, to obtain the preprocessed monitoring image.

4. The method according to claim 2, characterized in that, The security detection model includes a 3D convolutional module and an LSTM module; The three-dimensional convolution module is set between the backbone network and the feature pyramid of the YOLOv11-S network and is used to extract the spatiotemporal features of multiple consecutive monitoring images. The LSTM module is positioned in front of the detection head of the YOLOv11-S network and is used to predict the position of the target based on multiple consecutive frames of monitoring images, thereby obtaining a sequence of target positions for the next three frames.

5. The method according to claim 2, characterized in that, The sampling machine further includes an infrared camera connected to the control module; the preprocessing of the monitoring image to obtain the preprocessed monitoring image includes: The monitoring image and the infrared thermal imaging image are spatiotemporally aligned using timestamps and a cross-modal Transformer module to obtain an aligned image, which is then input into the security detection model. The cross-modal Transformer module includes a feature extraction submodule, an encoding submodule, a decoding submodule, and a residual connection submodule. The feature extraction submodule is used to extract features from the monitoring image and the infrared thermal imaging image respectively through a dual-branch convolutional neural network to obtain visible light feature map and infrared feature map; The encoding submodule is used to perform position encoding and multi-head self-attention calculation on the visible light feature map to obtain an enhanced visible light feature map; The decoding submodule is used to generate a query vector based on the enhanced visible light feature map, and to generate a key vector and a value vector based on the infrared feature map; and to determine the cross attention weights using scaled dot product attention to obtain the aligned infrared feature map. The residual connection submodule is used to add the aligned infrared feature map to the infrared feature map element by element to obtain the aligned feature map; The security detection model includes a spectral fusion module connected to the backbone submodule in the YOLOv11-S network; the spectral fusion module is used to dynamically allocate fusion weights for the visible light channel and the infrared channel.

6. The method according to claim 5, characterized in that, The sampling machine also includes a radar module connected to the control module; after inputting the preprocessed monitoring image into the trained safety detection model to identify vehicles and personnel within the sampling area and obtain the identification results, the process further includes: Acquire point cloud data from the radar module; The spatial position of the recognition result is verified based on the point cloud data to correct the image ranging error and obtain the verified recognition result.

7. The method according to claim 6, characterized in that, The sampling machine also includes an illumination detection module connected to the control module. The preprocessed monitoring image is input into a trained safety detection model to identify vehicles and personnel within the sampling area, obtaining identification results, including: Obtain illuminance information; Determining the detection mode and adjusting the preset distance threshold based on the illuminance information; specifically including: When the illuminance is greater than the second illuminance threshold, a pure visible light detection mode is adopted, and the preset distance threshold is D1; When the illuminance is greater than the first illuminance threshold and not greater than the second illuminance threshold, the infrared camera is activated and a dual-spectrum detection mode is adopted. The preset distance threshold is D1×b; where b is a preset multiple threshold, and the first illuminance threshold is less than the second illuminance threshold. When the illuminance is not greater than the first illuminance threshold, the radar module is activated and the radar ranging compensation mode is adopted. The preset distance threshold is D1×(1+c×(50-current illuminance)), where c is the illuminance compensation coefficient.

8. A safety detection device for sampling personnel in a sampling machine, characterized in that, include: The acquisition module is used to acquire monitoring images within the sampling area; The preprocessing module is used to preprocess the monitoring image to obtain the preprocessed monitoring image; The target detection module is used to input the preprocessed monitoring image into the trained security detection model, identify vehicles and personnel in the sampling area respectively, and obtain the identification results; A distance determination module is used to determine the distance information between the person and the vehicle based on the recognition result; The early warning module is used to compare the distance information with a preset distance threshold to determine whether the person is within a safe distance range; if not, an alarm mechanism is triggered.

9. A sampling machine, characterized in that, The system includes a control module, and a camera module, an infrared camera, a radar device, and an illuminance detection module, all connected to the control module. The control module is used to execute the sampling personnel safety detection method for the sampling machine as described in any one of claims 1-7.

10. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the sampling personnel safety detection method of any one of claims 1-7.

Images