Maintenance library personnel illegal behavior early warning method and system based on lightweight detection model

Abstract

The invention relates to the technical field of behavior detection, and discloses a maintenance library personnel illegal behavior early warning method and system based on a lightweight detection model. Preprocessing a target area image of the target locomotive maintenance library and labeling a dangerous high-incidence area label set and a behavior judgment label set as a dangerous scene data set and a behavior judgment data set; constructing an illegal behavior detection model composed of a backbone network, a feature fusion network and a detection head; giving out illegal behavior judgment result detection according to the target area image acquired in real time and the illegal behavior detection model, and outputting an illegal behavior type; and according to the violation behavior type, triggering a corresponding early warning mode to give an alarm, and recording violation event information. According to the invention, the operation scene of the locomotive personnel in the maintenance warehouse can be monitored in real time, and once the personnel behavior violation is found, the early warning signal can be sent out immediately, so that the management personnel can take measures in time, and accidents are avoided.

Description

Technical Field

[0001] The present invention relates to the field of behavior detection technology, and in particular to a method and system for early warning of illegal behavior of maintenance depot personnel based on a lightweight detection model. Background Art

[0002] With the continuous expansion of industrial production and increasing demands for productivity, locomotive maintenance depots are being used more frequently, and the number of operators is also increasing accordingly. The operating environment of locomotive maintenance depots is complex and presents numerous hazards. Traditional manual oversight methods are unable to meet the safety management needs of large-scale, complex production environments. Safety accidents caused by personnel violations during locomotive maintenance depot operations are also common, resulting in significant casualties and financial losses for enterprises.

[0003] Patent application number CN202110951679.2, "A Construction Site Safety Intelligent Early Warning Method and Device," provides an intelligent construction site safety early warning method and device, comprising the following steps: labeling objects in construction site images, acquiring model training data, adding an auxiliary positioning network (LAN) to the original YOLO-V3 detection model to obtain an improved YOLO-V3 detection model capable of detecting accurate classification and positioning information, training the improved YOLO-V3 detection model using the model training data, establishing a recognition information database, acquiring real-time construction site scene images, and detecting violations based on the improved YOLO-V3 detection model and the recognition information database and real-time construction site scene images. If violations are detected, information about the offending individuals is detected and pushed to the backend. This existing technology only detects and warns of violations involving personnel not wearing safety helmets on construction sites, but does not consider the different violations that occur in different scene areas. This makes it unsuitable for complex environments such as locomotive maintenance depots. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for early warning of illegal behaviors of maintenance depot personnel based on a lightweight detection model, which solves the problem of rapid real-time classification and monitoring of the behaviors of operating personnel in large-scale and long-distance scenarios in locomotive maintenance depots, and realizes timely early warning of illegal behaviors in different operating scenarios to avoid accidents.

[0005] The present invention adopts the following technical solution: a maintenance depot personnel violation warning method based on a lightweight detection model, comprising the following steps:

[0006] S1: Obtain the target area image of the target locomotive maintenance depot and perform preprocessing, then label the target area image according to the high-risk area label set to obtain the dangerous scene dataset;

[0007] S2: Obtain the target area image preprocessed in step S1, annotate the target area image according to the behavior determination label set, and obtain the behavior determination data set;

[0008] S3: Build a violation detection model consisting of a backbone network, a feature fusion network, and a detection head, and use the dangerous scene dataset and behavior judgment dataset to train and verify the violation detection model;

[0009] S4: Based on the real-time acquired target area image, a violation detection model is used to generate a violation determination result and output the violation type;

[0010] S5: Trigger the corresponding warning method according to the type of violation and record the violation event information.

[0011] Furthermore, based on the high-risk area label set including ditch labels and roof labels, the target area image is classified and labeled into target sub-areas including ditch area, roof area and working area. The working area is the working area outside the ditch area and roof area in the target area; based on the behavior judgment label set including coupler labels, helmet labels, seat belt labels, human head labels, human foot labels and torso labels, the target area image is labeled by generating target bounding boxes on these six types of labels.

[0012] Furthermore, the violation detection model has the following features:

[0013] S310: Optimize the backbone network of the original detection model by replacing regular convolutions with depthwise separable convolution layers;

[0014] S320: Adding a cross-layer connection mechanism to the original detection model after optimizing the backbone network to optimize the feature fusion network;

[0015] S330: Inserting a temporal behavior analysis layer before the detection head in the original detection model after optimizing the feature fusion network.

[0016] Furthermore, the original detection model is built based on the yolov4 detection model, with CSP-DarkNet53 as the backbone network of the original yolov4 detection model. The 3×3 convolutional layers in the last three stages of CSP-DarkNet53 are replaced by depthwise separable convolution layers. The depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution. The feature fusion network adopts the feature pyramid network FPN-PANet structure, and inserts the ConvLSTM layer before the detection head as the temporal behavior analysis layer.

[0017] Furthermore, adding a cross-layer connection mechanism in step S320 includes the following steps:

[0018] S3201: Build cross-layer connections to inject low-level high-resolution feature maps directly into the top-level feature maps after the PANet path is passed from bottom to top;

[0019] S3202: Add a small object perception layer to the pyramid network FPN-PANet.

[0020] Furthermore, the violation detection model is trained and verified: the clustering results are calculated and output by applying the K-means clustering algorithm, the clustering results are classified and positioned, the corresponding positioning information and confidence score are obtained, and it is determined whether the confidence score is higher than the threshold. If the confidence score is higher than the threshold, it is determined that the reference target frame contains the coupler label, helmet label, seat belt label, human head label, human foot label and torso label.

[0021] Furthermore, the violation determination logic is set as follows based on the tag's location information and confidence score:

[0022] In N consecutive frames of real-time target area images, if the target torso tag is detected in the roof area and the seat belt tag of the same target person is not detected, the person is judged to have violated the rules by not wearing a seat belt in the roof area.

[0023] When the target human foot tags are detected on both sides of the ditch area in N consecutive frames of real-time target area images, the violation of "personnel crossing the ditch" is determined;

[0024] In N consecutive frames of real-time target area images, if the target human head tag is detected in the working area but the safety helmet tag is not detected, the person is judged to have violated the rules by "not wearing a safety helmet in the working area";

[0025] In the continuous N-frame real-time target area image, when the calculated actual depth distance ΔD between the target coupler and the target body is always less than the safety distance T safe , and determined that the personnel violation was “the personnel was directly under the coupler”.

[0026] Furthermore, the process of determining the personnel violation of "personnel located directly under the coupler" is as follows:

[0027] The camera's internal and external parameters are obtained through camera calibration algorithms such as the checkerboard method. The actual approximate distances from the coupler and the person to the camera are calculated based on the pixel heights of the coupler label and torso label bounding boxes, the actual heights of the coupler and the human torso, and the camera's internal and external parameters. When the difference in the horizontal coordinates of the centers of the coupler label and torso label bounding boxes is less than a preset threshold, the actual depth spacing between the coupler and the human torso is calculated based on the actual approximate distance and the angle between the line connecting the coupler and the human body and the optical axis. When the actual depth spacing is less than the safety distance, it is determined that the person has violated the regulations by being "directly below the coupler."

[0028] The maintenance depot personnel violation warning system based on lightweight detection model includes: classification and judgment unit, real-time data acquisition module and warning module; among them,

[0029] Real-time data acquisition module: It is set at a preset position in the target maintenance warehouse and covers the target sub-area. It is used to obtain the target area image in the target sub-area in real time according to the preset sampling frequency and output the target area image to the classification and judgment unit;

[0030] Classification and determination unit: Based on the received target area image, the unit identifies the target sub-areas based on the preset violation detection model, detects personnel violations based on the identified target sub-areas, assesses the type of personnel violations based on the violation determination logic in the violation detection model, and transmits the violation type to the early warning module. The classification and determination unit includes an area classification module and a behavior determination module.

[0031] Early warning module: triggers the corresponding early warning method according to the type of violation and records the violation event information.

[0032] Furthermore, the classification determination unit includes a region classification module and a behavior determination module;

[0033] Region classification module: Based on the preset violation detection model, the target area image is classified according to the high-risk area label set to identify the target sub-region;

[0034] Behavior determination module: detects personnel violations and determines the type of personnel violations based on the violation determination logic in the violation detection model.

[0035] The present invention performs real-time early warning detection on illegal behaviors of personnel during operations in the maintenance depot. Through the optimized target detection model, it monitors illegal operations in the operation process in real time and issues timely early warnings to eliminate potential safety hazards that may be caused by illegal behaviors during the operation process and improve the safety of operations.

[0036] In a locomotive maintenance depot environment with a long distance and large scenes, a large amount of information needs to be processed. The violation detection model constructed replaces conventional convolution with a depthwise separable convolution layer to reduce the number of downsampling times, so as to retain more small target details and the resolution of high-level feature maps is relatively low. This decomposition method greatly reduces the amount of computation required for the model, which can improve the model's operating efficiency, enabling it to process images faster and achieve real-time detection. By adding a cross-layer connection mechanism, while ensuring the accuracy of detecting large targets such as ditches and roofs, the detection accuracy of small target labels is also improved. A temporal feature fusion layer is inserted before the detection head to enhance the accuracy of detecting violations and avoid accidental errors in single-frame detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Flowchart of the method for early warning of illegal behavior of maintenance depot personnel based on the lightweight detection model provided by the present invention;

[0038] Figure 2 This is a block diagram of the maintenance depot personnel violation warning system based on the lightweight detection model provided by the present invention;

[0039] Figure 3 This is a schematic diagram of the detailed functional modules of the classification and judgment unit in the maintenance depot personnel violation warning system provided by the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of this document clearer, the technical solutions in the embodiments of this document will be clearly and completely described below in conjunction with the drawings in the embodiments of this document. Obviously, the described embodiments are part of the embodiments of this document, not all of the embodiments. Based on the embodiments of this document, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this document. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of this document can be combined with each other in any way.

[0041] Common target detection models have large computational complexity and slow running speed, making it difficult to meet real-time requirements. During training, they do not fully consider the special features within the locomotive maintenance depot, resulting in poor results in detecting violations by personnel within the locomotive maintenance depot. In addition, common target detection models often have low detection accuracy for small-scale targets, making it easy to miss detections or inaccurate positioning; and for large-scale targets, it may be difficult to accurately classify them because the features are too complex.

[0042] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0043] As attached Figure 1 As shown, an exemplary embodiment provided by the present invention is a method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model, comprising the following steps:

[0044] S1: Obtain the target area image of the target locomotive maintenance depot and perform preprocessing, then label the target area image according to the high-risk area label set to obtain the dangerous scene dataset;

[0045] Labeling tools such as LabelImg are used to classify the polygon outlines of the high-risk area labels in the image, namely the ditch label and the roof label, to obtain a dangerous scene dataset, which is used to divide the target area into several target sub-areas; the target sub-areas include the ditch area, the roof area and the working area.

[0046] According to an exemplary embodiment, a plurality of cameras facing different directions are used to cover the target maintenance depot, and recorded videos of the personnel behavior in the locomotive maintenance depot are obtained through the cameras. Each video is extracted frame by frame to generate a self-photographed scene picture of the locomotive maintenance depot, which is combined with the public scene pictures of the locomotive maintenance depot available to the public to form a training set to ensure the richness of the obtained scenes. The locomotive maintenance depot includes a trench area, a roof area and an operating area; the trench refers to an underground engineering structure on the ground located on the inner side of the track gauge of the locomotive depot inspection line or the station where the locomotive depot is located. Steps are provided on both sides of the trench for maintenance personnel or crew members to enter, so as to facilitate the inspection of the running part or frame of the locomotive. Part of the inspection and maintenance; the roof area refers to the top area of ​​the train set in the locomotive maintenance depot, which is used by staff to conduct performance and appearance tests on the train set roof; the working area refers to the working area outside the ditch area and the roof area in the locomotive maintenance depot; the preprocessing operations include: using non-local mean denoising to reduce the noise caused by metal reflection and equipment vibration; using adaptive histogram equalization to enhance the details of low-light areas and balance the uneven lighting; by scaling the target area image to a fixed size, maintaining the aspect ratio and filling the edges with grayscale, the target area image is normalized to [0,1] to adapt to the input requirements of the violation detection model.

[0047] S2: Obtain the target area image preprocessed in step S1, annotate the target area image according to the behavior determination label set, and obtain the behavior determination data set;

[0048] The target area image is annotated by using labeling tools such as LabelImg according to the behavioral judgment label set including coupler label, helmet label, seat belt label, human head label, human foot label and torso label. The annotation method is to generate a target bounding box on these six types of labels and save the annotation file as positioning information. The obtained positioning information includes the center point coordinates of the target bounding box, the width and height relative to the image size ratio; the positioning information of the torso label is (x p ,y p ,w p ,h p );x p Indicates the horizontal coordinate of the center point of the torso label bounding box, y p Represents the vertical coordinate of the center point of the torso label bounding box, w p The pixel width of the torso label bounding box, h p represents the pixel height of the torso label bounding box; p represents the reference of the torso label; the positioning information of the coupler label is (x g ,y g ,w g ,h g ), x g ,yg ,w g and h g They represent the horizontal coordinate, vertical coordinate, pixel width and pixel height of the center point of the coupler label boundary box respectively; g represents the reference of the coupler label;.

[0049] S3: Build a violation detection model consisting of a backbone network, a feature fusion network, and a detection head, and use the dangerous scene dataset and behavior judgment dataset to train and verify the violation detection model;

[0050] The construction of the violation detection model includes the following steps:

[0051] S310: Replace the conventional convolution in the backbone network with a depth-wise separable convolution layer to optimize the original detection model;

[0052] The violation detection model is built based on the Yolov4 detection model, using CSP-DarkNet53 as the backbone network of the original Yolov4 detection model, retaining its high-level feature extraction capabilities. The original Yolov4 detection model suffers from frame rate drops and poor real-time performance when processing high-resolution images such as locomotive maintenance depot monitoring images with large data volumes.

[0053] According to an exemplary embodiment, the conventional convolution for downsampling and preliminary feature extraction in CSP-DarkNet53 is replaced with a depthwise separable convolution layer. In this embodiment, the 3×3 convolution layers in the last three stages of CSP-DarkNet53 are replaced with depthwise separable convolution layers to balance computational efficiency and feature expression capabilities. The depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, providing a multi-scale feature map as input for the feature fusion network. The feature map levels output by the backbone network are P1, P2, P3, P4, and P5. A smaller convolution kernel of 3×3 is used instead of 5×5 for the underlying convolution layer in order to reduce the number of downsampling times and retain more small target detail information. The high-level feature map has a lower resolution. This decomposition method greatly reduces the computational complexity of the model, improves the operating efficiency of the model, and enables it to process images faster and achieve real-time detection.

[0054] Depthwise separable convolution decomposes the standard convolution into depthwise convolution, the formula is as follows:

[0055]

[0056] in, Represents the output value of the cth channel at position (i, j) after depth convolution; dw represents the reference of depth convolution; c represents the channel index of the input feature map (c=1,2,...,C in ); K represents the convolution kernel size; represents the depth convolution kernel weight of the c-th channel, position (k, l); X (c) Represents the cth channel of the input feature map; (i, j) represents the spatial position coordinates of the output feature map; Indicates the offset of the center position of the convolution kernel;

[0057] Depthwise separable convolution also decomposes the standard convolution into point-wise convolution, as follows:

[0058]

[0059] in, Represents the output value of the d-th channel at position (i, j) after point-by-point convolution; pw represents the reference of point-by-point convolution; Represents the weight value between the d-th channel and the c-th channel in the point-by-point convolution kernel; Represents the value of the c-th channel at position (i, j) after depth convolution;

[0060] S320: Adding a cross-layer connection mechanism to the original detection model after optimizing the backbone network to optimize the feature fusion network;

[0061] According to an exemplary embodiment, in a locomotive maintenance depot scenario, the feature map levels output by the backbone network are P1, P2, P3, P4, and P5, where P1 and P2 are low-level, high-resolution feature map levels corresponding to target details such as edges, contours, colors, and textures, with rich features but weak semantics; P4 and P5 are high-level, low-resolution feature map levels corresponding to semantic features, with strong semantics but lost details; the middle feature map level P3 is set as a mid-level feature that takes into account both details and semantics;

[0062] By counting the sizes of different target labels in the training data, determining the pixel ratio range of the target label in the image, setting the ratio of the height or width of the small target label to the image size, and taking the target label with a ratio of less than 5% as the small target label, using tools such as Grad-CAM to visualize the model's focus area on small targets, and obtaining the feature map level that is more critical for small target detection. In the present invention, the low-level high-resolution feature map level is more suitable for detecting small targets. The small target labels include coupler labels, helmet labels, seat belt labels, human head labels and human foot labels.

[0063] The feature fusion network adopts the feature pyramid network FPN-PANet structure; FPN is a top-down feature pyramid that fuses high-level semantic features including target label categories and violation identification with low-level detail features including position, shape, and color layer by layer through a top-down path, solving the problem that traditional single-scale feature maps are insufficient to detect small targets; PANet is a bottom-up path aggregation that adds bottom-up path aggregation on the basis of FPN, further enhancing the interaction between adjacent layer features and shortening the information flow path between low-level detail features and high-level semantic features.

[0064] S3201: Build cross-layer connections and inject the low-level high-resolution feature map C2 directly into the top-level feature map N5 after the PANet path is passed from bottom to top to obtain the final fused feature map The process is as follows:

[0065]

[0066] Among them, Conv 1×1 Indicates the application of a 1×1 convolution operation for channel alignment; C2 represents the low-level high-resolution feature map output by the backbone network CSPDarknet53; N5 represents the top-level feature map after the PANet path is passed from bottom to top; UpSample ×8 () indicates 8x upsampling operation; Indicates channel splicing; N ′ 5 represents the initial feature map after splicing.

[0067] Calculate the adaptive fusion weight coefficient,

[0068] α=σ(MLP(GAP(N ′ 5))) (4)

[0069] Among them, α represents the adaptive fusion weight coefficient; σ() represents the Sigmoid activation function; MLP() represents the nonlinear transformation of each image block, which is used to learn the weight relationship between low-level details and high-level semantics; GAP() represents the global average pooling operation, which is used to take the spatial average of the feature map of each channel.

[0070]

[0071] in, Represents the final feature map after adaptive fusion.

[0072] Building cross-layer skip connections enables the underlying high-resolution small target features to be more directly integrated with high-level semantic features, thereby improving the small target detection performance, increasing the feature output of the corresponding level of small targets, and enhancing the model's perception of small targets.

[0073] S3202: Add a small object perception layer to the pyramid network FPN-PANet;

[0074] The process is as follows:

[0075]

[0076] Among them, N3 represents the existing middle-level feature map in FPN-PANet, UpSample() represents the upsampling operation, which is used to upsample N3 to the same resolution as C2; N2 represents the middle-level feature map output by the newly added small target perception layer in FPN-PANet.

[0077] Although adding the small object perception layer results in an increase in computational complexity, replacing conventional convolution with depthwise separable convolution in step S310 can offset the performance loss.

[0078] The low-level high-resolution feature map C2 output by the backbone network CSPDarknet53 is used to obtain small areas such as the edge of the coupler, the edge of the person's feet, the person's head, and the edge of the seat belt; the existing mid-level feature map N3 in FPN-PANet is used to identify the association semantics between the high-risk area label set and the behavior judgment label set.

[0079] Calculate the loss function formula of the detection head,

[0080] L Slayer =δ×L CIoU (7)

[0081] Among them, L Slayer Represents the total loss value of the small target perception layer; L CIoU represents the complete IoU loss function; δ represents the loss weight coefficient.

[0082] According to an exemplary embodiment, small target detection is difficult, and the gradient signal is amplified by loss weight; in the locomotive maintenance depot dataset, 1.5 to 2.0 times the weight can effectively improve the average precision (AP) detection of small target labels such as helmets.

[0083] S330: inserting a temporal behavior analysis layer before the detection head in the original detection model after optimizing the feature fusion network;

[0084] Insert the ConvLSTM layer before the detection head and continuously input N frames of features to judge the violation behavior of personnel.

[0085] (H t ,C t )=ConvLSTM(F t ,H t-1 ,C t-1) (8)

[0086] Among them, F t represents the multi-scale feature map from FPN-PANet at frame t, t = 1, ..., N, where N is the preset number of frames used to judge violations; H t-1 Indicates the hidden state of the previous frame t-1 that carries historical timing information; C t-1 Represents the cell state of the long short-term memory carrier of the previous frame LSTM; H t Represents the hidden state at the current moment, which serves as the input of the subsequent detection head; C t Represents the cell state of the current frame, which is passed to the next frame.

[0087] Used to analyze target movement trajectories for security risk behavior. The original YOLOv4 target detection model only processes single-frame images and ignores the temporal correlation of targets. ConvLSTM combines the temporal modeling capabilities of LSTM with convolution operations to learn the dynamic changes of targets and the movement trajectories of people.

[0088] Training and validating the violation detection model using the dangerous scene dataset and behavior judgment dataset includes the following process:

[0089] The dangerous scene dataset and behavior judgment dataset are divided into training set, validation set and test set in a ratio of 8:1:1 respectively. Based on the training set, the violation detection model is trained to obtain the trained violation detection model. The detection performance of the violation detection model is verified based on the validation set. Finally, based on the test set, violation detection is performed on the violation detection model that has passed the detection performance verification to confirm the effectiveness of the trained violation detection model.

[0090] The violation detection model marks the ditch area and roof area in the test set scene image, and marks the part of the target maintenance warehouse that does not belong to the ditch area and roof area as the working area. Through target area segmentation, the target area is divided into target sub-areas including the ditch area, roof area and working area.

[0091] According to an exemplary embodiment, the image data of the locomotive maintenance depot is relatively limited. By segmenting the target area, the U-Net algorithm is used to segment the target area, which has good performance. At the same time, the U-Net algorithm has high segmentation accuracy and can accurately divide the boundaries of different areas. It can also better identify relatively narrow areas such as ditches that have significant differences in characteristics from the surrounding environment.

[0092] The monitoring range of each sub-area is relatively narrowed, which allows for more focus on targets within a specific area and reduces interference from irrelevant information, thereby more accurately identifying and tracking the behavior of locomotives and personnel within the area. The operating behaviors in different sub-areas have different characteristics. Through target area segmentation, the scope of the ditch area, roof area, and operating area can be set in advance.

[0093] According to an exemplary embodiment, the violation detection model calculates the movement trajectory of the target person by applying the K-means clustering algorithm, outputs the clustering results, performs classification tasks and positioning tasks on the clustering results, and obtains corresponding confidence scores.

[0094] S4: Based on the real-time acquired target area image, a violation detection model is used to generate a violation determination result and output the violation type;

[0095] According to an exemplary embodiment, a plurality of cameras facing different directions are used to cover a target maintenance depot, and real-time behavior videos of personnel in target areas of multiple locomotive maintenance depots are obtained through the cameras; the real-time streaming protocol RTSP is used to transmit the real-time behavior videos from the camera to the server; OpenCV is used to process the RTSP push stream to obtain a real-time target area image and perform real-time deblurring and contrast enhancement, and adjust the resolution to scale the image to the model input size.

[0096] In addition, according to an exemplary embodiment, a multi-reference object comparison method is used to determine whether the camera angle has been artificially offset to determine whether the sub-area has changed, so as to avoid failure of the monitoring range; a fixed reference object is selected at a fixed position within the camera's field of view as a benchmark for angle offset detection; the reference object can be a placed calibration plate or a fixed marker, or it can be a building or tool that is not easily changed within the camera's field of view; the position coordinates of the reference object in the image are extracted, and a reference coordinate system is established based on the initial calibration position. The reference object coordinate offset is calculated by Euclidean distance or angle deviation to determine whether the camera has been offset; an offset threshold is set, and when the reference object coordinate offset exceeds the offset threshold, an alarm is triggered and the monitoring function is suspended, and manual calibration is performed.

[0097] The target area image in the target sub-area is acquired in real time according to the preset sampling frequency, and the target sub-area is segmented from the target area image through the PANet structure in the violation detection model; the predefined area is marked according to the label set information of the high-risk area, the ditch area is marked as a polygon coordinate set, including the left and right boundaries, and the area extending 10 pixels to the left and right of the ditch; the roof area is marked as a polygon coordinate set; and the working area is marked as a polygon coordinate set.

[0098] The violation detection model performs violation detection on the target sub-area, outputs a clustering result, performs classification tasks and positioning tasks on the clustering result, and obtains positioning information and a confidence score; determines whether the confidence score is higher than a threshold, and if the confidence score is higher than the threshold, determines that the reference target frame contains the coupler label, helmet label, seat belt label, human head label, human foot label, and torso label.

[0099] According to an exemplary embodiment, the threshold value may be set to 0.6.

[0100] According to an exemplary embodiment, the violation determination logic set based on the tag's location information and confidence score is as follows:

[0101] In N consecutive frames of real-time target area images, if the target torso tag is detected in the roof area and the seat belt tag of the same target person is not detected, the person is judged to have violated the rules by not wearing a seat belt in the roof area.

[0102] When the target human foot tags are detected on both sides of the ditch area in N consecutive frames of real-time target area images, the violation of "personnel crossing the ditch" is determined;

[0103] In N consecutive frames of real-time target area images, if the target human head tag is detected in the working area but the safety helmet tag is not detected, the person is judged to have violated the rules by "not wearing a safety helmet in the working area";

[0104] In the continuous N-frame real-time target area image, when the calculated actual depth distance ΔD between the target coupler and the target body is always less than the safety distance T safe , and determined that the personnel violation was “the personnel was directly under the coupler”.

[0105] According to an exemplary embodiment, the preset sampling frequency is set to 5 fps, and N is set to 10.

[0106] According to an exemplary embodiment, the internal and external parameters of the camera are obtained by a camera calibration algorithm such as a checkerboard method, and the actual height H of the coupler is pre-calibrated. g Meters, average height of personnel (torso) H p rice.

[0107] Calculate the actual approximate distance from the coupler to the camera:

[0108]

[0109] The actual approximate distance from the person (torso) to the camera:

[0110]

[0111] Where f represents the internal and external parameters of the camera; H g Indicates the actual height of the coupler (in meters); h g Indicates the pixel height of the coupler label bounding box (in pixels); Scale indicates the conversion factor from pixels to meters; H p Indicates the actual height of the person's torso (in meters); h p Indicates the pixel height of the torso label bounding box (in pixels).

[0112] When the horizontal coordinate difference between the center of the bounding box of the coupler label and the torso label |Δu| <δ, it means that the coupler and the person are adjacent to each other in the image. Calculate the actual depth distance ΔD between the target coupler and the target torso:

[0113] |Δu|=|x g -x p ∣ (11)

[0114] ΔD=|D g -D p ∣*cosθ (12)

[0115] Among them, δ represents the threshold of the horizontal coordinate difference between the center of the bounding box of the coupler label and the torso label; x g Represents the horizontal coordinate of the center point of the torso label bounding box;

[0116] Estimation of the angle between the coupler and the operator and the optical axis through stereo vision or multi-view fusion:

[0117]

[0118] If ΔD <T safe , it is determined that "a person is directly under the coupler"; an embodiment is provided to set a safety distance T safe is 1.0m.

[0119] If the same person triggers multiple violation conditions, the violation with the highest risk level will be output. The risk levels of the violation are processed from high to low as follows: the person is directly under the coupler, the person is not wearing a seat belt in the roof area, the person is crossing the ditch, and the person is not wearing a safety helmet in the working area.

[0120] According to an exemplary embodiment, if it is detected that the same person has committed two violations at the same time, namely, not wearing a safety helmet in the working area and being located directly under the coupler, the latter violation will be given priority.

[0121] S5: Trigger the corresponding warning method according to the type of violation and record the violation event information.

[0122] According to an exemplary embodiment, an early warning mechanism is automatically triggered according to the type of violation, and a violation notification is sent to the project manager through sound, light signals or directly, and the violation event information including the violation time and violation image is recorded so that timely measures can be taken; for example, the violation is announced through radio broadcasting to promptly remind the violator to correct it.

[0123] This embodiment adopts an optimized target detection model, which can analyze and process a large amount of monitoring data in a short period of time, quickly detect violations, and meet real-time requirements. It introduces depthwise separable convolution to balance computational efficiency and feature expression capabilities. While ensuring the accuracy of detecting large targets such as ditches and roofs, it improves the detection accuracy of small target labels by adding a cross-layer connection mechanism. It also continuously analyzes N-frame features by inserting a temporal feature fusion layer before the detection head, capturing the coherent movements of personnel in the locomotive maintenance depot, enhancing the accuracy of detecting violations, and avoiding errors in single-frame detection.

[0124] As attached Figure 2 As shown, an exemplary embodiment provided by the present invention is a maintenance depot personnel violation warning system based on a lightweight detection model, comprising: a classification and determination unit, a real-time data acquisition module and an early warning module; wherein,

[0125] Real-time data acquisition module: It is set at a preset position in the target maintenance warehouse and covers the target sub-area. It is used to obtain the target area image in the target sub-area in real time according to the preset sampling frequency and output the target area image to the classification and judgment unit;

[0126] Classification and determination unit: Based on the received target area image, the unit identifies the target sub-areas based on the preset violation detection model, detects personnel violations based on the identified target sub-areas, assesses the type of personnel violations based on the violation determination logic in the violation detection model, and transmits the violation type to the early warning module. The classification and determination unit includes an area classification module and a behavior determination module.

[0127] Early warning module: triggers the corresponding early warning method according to the type of violation and records the violation event information.

[0128] As attached Figure 3 As shown, the classification determination unit includes an area classification module and a behavior determination module;

[0129] Region classification module: Based on the preset violation detection model, the target area image is classified according to the high-risk area label set to identify the target sub-region;

[0130] Behavior determination module: detects personnel violations and determines the type of personnel violations based on the violation determination logic in the violation detection model.

[0131] The violation judgment logic in the violation detection model is:

[0132] In N consecutive frames of real-time target area images, if the target torso tag is detected in the roof area and the seat belt tag of the same target person is not detected, the person is judged to have violated the rules by not wearing a seat belt in the roof area.

[0133] When the target human foot tags are detected on both sides of the ditch area in N consecutive frames of real-time target area images, the violation of "personnel crossing the ditch" is determined;

[0134] In N consecutive frames of real-time target area images, if the target human head tag is detected in the working area but the safety helmet tag is not detected, the person is judged to have violated the rules by "not wearing a safety helmet in the working area";

[0135] In the continuous N-frame real-time target area image, when the calculated actual depth distance ΔD between the target coupler and the target body is always less than the safety distance T safe , and determined that the personnel violation was “the personnel was directly under the coupler”.

[0136] This embodiment can monitor the operation scenes of locomotive personnel in the maintenance depot in real time. Once any behavioral violations are found, an early warning signal can be immediately issued, so that management personnel can take timely measures to avoid accidents. By adopting the optimized target detection model, various behavioral violations can be accurately identified, reducing misjudgments and missed judgments. The feature fusion network optimization module can further improve the accuracy of small target detection. The behavioral violation early warning detection of locomotive personnel in the maintenance depot can monitor the illegal operation behaviors in real time during the operation process and issue timely warnings, eliminating the safety hazards that may be caused by illegal behaviors during the operation process, and improving the safety of the operation.

Claims

1. A maintenance depot personnel violation warning method based on a lightweight detection model is characterized by: The steps include: S1: Obtain the target area image of the target locomotive maintenance depot and perform preprocessing, then label the target area image according to the high-risk area label set to obtain the dangerous scene dataset; S2: Obtain the target area image preprocessed in step S1, annotate the target area image according to the behavior determination label set, and obtain the behavior determination data set; S3: Build a violation detection model consisting of a backbone network, a feature fusion network, and a detection head, and use the dangerous scene dataset and behavior judgment dataset to train and verify the violation detection model; S4: Based on the real-time acquired target area image, a violation detection model is used to generate a violation determination result and output the violation type; S5: Trigger the corresponding warning method according to the type of violation and record the violation event information.

2. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 1 is characterized in that: According to the high-risk area label set including ditch labels and roof labels, the target area image is classified and labeled into target areas including ditch area, roof area and working area. The working area is the working area outside the ditch area and roof area in the target area; according to the behavior judgment label set including coupler labels, helmet labels, seat belt labels, human head labels, human foot labels and torso labels, the target area image is labeled by generating target bounding boxes on these six types of labels.

3. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 1 is characterized in that: The backbone network adopts a depth-separable convolution layer, and optimizes the feature fusion network through a cross-layer connection mechanism. A temporal behavior analysis layer is also set before the detection head.

4. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 3 is characterized in that: The violation detection model is built based on the Yolov4 detection model, with CSP-DarkNet53 as the backbone network of the original Yolov4 detection model. The convolutional layers in the last three stages of CSP-DarkNet53 are depthwise separable convolution layers. Depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution. The feature fusion network adopts the feature pyramid network FPN-PANet structure, and inserts the ConvLSTM layer before the detection head as the temporal behavior analysis layer.

5. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 4 is characterized in that: The steps described above optimize the feature fusion network through a cross-layer connection mechanism, including the following steps: S3201: Build cross-layer connections to inject low-level high-resolution feature maps directly into the top-level feature maps after bottom-up transmission of the PANet path to obtain the final fused feature map; S3202: Add a small target perception layer to the pyramid network FPN-PANet and output the middle-level feature map generated by the small target perception layer.

6. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 3 is characterized in that: Train and verify the violation detection model: calculate and output the clustering results by applying the K-means clustering algorithm, perform classification tasks and positioning tasks on the clustering results, obtain corresponding positioning information and confidence scores, and determine whether the confidence scores are higher than a threshold. If the confidence scores are higher than the threshold, determine that the coupler label, helmet label, seat belt label, human head label, human foot label, and torso label are included in the reference target frame.

7. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 2 is characterized in that: The violation determination logic is set as follows based on the tag's location information and confidence score: In N consecutive frames of real-time target area images, if the target torso tag is detected in the roof area and the seat belt tag of the same target person is not detected, the person is judged to have violated the rules by "not wearing a seat belt in the roof area"; If the target human foot tags are detected on both sides of the ditch area in N consecutive frames of real-time target area images, the violation of "person crossing the ditch" is determined; In N consecutive frames of real-time target area images, if the target human head tag is detected in the work area but the safety helmet tag is not detected, the person is judged to have violated the rules by "not wearing a safety helmet in the work area"; In the continuous N-frame real-time target area image, when the calculated actual depth distance between the target coupler and the target body is Always less than the safe distance , and determined that the personnel’s violation was “the personnel was directly under the coupler”.

8. The method for warning of illegal behavior of maintenance depot personnel based on a lightweight detection model according to claim 7 is characterized in that: The process for determining the personnel violation "personnel located directly under the coupler" is as follows: The camera's internal and external parameters are obtained through camera calibration algorithms such as the checkerboard method. The actual approximate distances from the coupler and the person to the camera are calculated based on the pixel heights of the coupler label and torso label bounding boxes, the actual heights of the coupler and the human torso, and the camera's internal and external parameters. When the difference in the horizontal coordinates of the center of the coupler label and torso label bounding boxes is less than a preset threshold, the actual depth separation between the coupler and the human torso is calculated based on the actual approximate distance and the angle between the line connecting the coupler and the human body and the optical axis. When the actual depth separation is less than the safe distance, the person is judged to have violated the regulations by being "directly below the coupler." 9. A maintenance depot personnel violation warning system based on a lightweight detection model, applicable to the maintenance depot personnel violation warning method based on a lightweight detection model as described in any one of claims 1 to 8, characterized in that: include: Classification and judgment unit, real-time data acquisition module and early warning module; among them, Real-time data acquisition module: It is set at a preset position in the target maintenance warehouse and covers the target sub-area. It is used to obtain the target area image in the target sub-area in real time according to the preset sampling frequency and output the target area image to the classification and judgment unit; Classification and determination unit: Based on the received target area image, the unit identifies the target sub-areas based on the preset violation detection model, detects personnel violations based on the identified target sub-areas, assesses the type of personnel violations based on the violation determination logic in the violation detection model, and transmits the violation type to the early warning module. The classification and determination unit includes an area classification module and a behavior determination module. Early warning module: triggers the corresponding early warning method according to the type of violation and records the violation event information.

10. The maintenance depot personnel violation warning system based on a lightweight detection model according to claim 9 is characterized by: The classification determination unit includes an area classification module and a behavior determination module; Region classification module: Based on the preset violation detection model, the target area image is classified according to the high-risk area label set to identify the target sub-region; Behavior determination module: detects personnel violations and determines the type of personnel violations based on the violation determination logic in the violation detection model.

Images