Method for detecting and tracking engineering vehicle in farmland in sheltered environment

Abstract

The invention discloses a method for detecting and tracking an engineering vehicle in a farmland in a sheltered environment, and relates to the technical field of computer vision and cultivated land protection, and the method comprises the following steps: S1, collecting and marking image data of the farmland engineering vehicle under different sheltering degrees, constructing a training set, a verification set and a test set, and improving and training a YOLOv8 model, constructing a high-precision engineering vehicle detection model, and constructing a data set preparation and model training module; s2, acquiring a farmland monitoring video stream in real time by using a high-definition camera, processing a video frame by using the improved YOLOv8 model trained in the step S1, outputting an engineering vehicle target detection frame with confidence, and constructing a video stream input and target detection module; according to the method for detecting and tracking the engineering vehicle in the farmland in the sheltered environment, the BiFPN and CBAM modules are introduced, so that the feature extraction capability of the model on sheltered and small-target engineering vehicles is remarkably enhanced, and the detection precision (mAP at 0.5) is improved to 87% or above in a sheltered scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and farmland protection, and particularly relates to a method for detecting and tracking engineering vehicles in farmland under shielding environment. BACKGROUND

[0002] At present, it is known that engineering vehicles are easily shielded by crops such as corn and rice after entering farmland, resulting in incomplete extraction of vehicle features (such as cab and wheels); when the engineering vehicles are shielded, the detection box is lost or deviated, resulting in invalid appearance feature matching; it is unable to distinguish whether the engineering vehicle is a legal operation (such as agricultural machinery) or an illegal intrusion, resulting in no recognition ability for the abnormal event of "engineering vehicles entering farmland" and being unable to trigger early warning; no specific environmental data of farmland is included, and the generalization ability of the model is weak in this scene, and the detection and tracking performance is significantly affected by the environment. SUMMARY

[0003] The present application aims to provide a method for detecting and tracking engineering vehicles in farmland under shielding environment, so as to solve the above-mentioned deficiencies in the prior art.

[0004] In order to achieve the above-mentioned purpose, the present application provides the following technical scheme: a method for detecting and tracking engineering vehicles in farmland under shielding environment, comprising the following steps: S1, collecting and labeling farmland engineering vehicle image data under different shielding degrees, constructing a training set, a validation set and a test set, improving and training a YOLOv8 model at the same time, constructing a high-precision engineering vehicle detection model, and constructing a data set preparation and model training module; S2, using a high-definition camera to obtain a farmland monitoring video stream in real time, and using the improved YOLOv8 model trained in step S1 to process the video frames, outputting an engineering vehicle target detection box with confidence, and constructing a video stream input and target detection module; S3, inputting the detection box obtained in step S2 into the improved DeepSort tracking algorithm, predicting the trajectory through Kalman filtering, and using an appearance feature extractor based on EfficientNet-B0 for data association, realizing stable tracking and identity ID maintenance of the engineering vehicle under shielding conditions, and constructing a target tracking and trajectory management module; S4, based on the pre-defined farmland prohibited entry area, analyzing the engineering vehicle motion trajectory obtained in step S3 in real time; when it is determined that the engineering vehicle has an illegal intrusion behavior, triggering an early warning mechanism immediately and pushing alarm information through a remote communication module, and constructing an illegal intrusion judgment and real-time multi-level early warning module.

[0005] Preferably, the engineering vehicle detection model introduces a bidirectional feature pyramid network in the YOLOv8 model, realizing multi-scale feature fusion; and integrates channel attention and spatial attention double mechanisms in the key feature layer of the backbone network.

[0006] Preferably, the bidirectional feature pyramid network comprises: a top-down feature transmission path for transmitting high-level semantic information to low-level features; a bottom-up feature transmission path for transmitting low-level detail information to high-level features; and a learnable feature weight distribution layer for optimizing the cross-scale feature fusion effect.

[0007] Preferably, the channel attention and spatial attention dual mechanism comprises: a channel attention module for generating channel weights through global pooling operation of feature maps; a spatial attention module for generating spatial weights through feature aggregation in the channel dimension; and the two-level attention weights acting on the input feature maps in turn.

[0008] Preferably, the improved DeepSort tracking algorithm in the step S3 comprises: using a pre-trained EfficientNet-B0 network as an appearance feature extractor and fine-tuning it for engineering vehicle images in a farmland occlusion scene; and setting an adaptive matching threshold: when the detection box confidence is lower than a preset threshold, the matching threshold of data association is automatically reduced to preferentially ensure the continuity of the track after temporary occlusion.

[0009] Preferably, the adaptive matching threshold is specifically: maintaining the default matching threshold of 0.6 when there is no occlusion; adjusting the matching threshold to 0.4-0.5 when the occlusion is 30%-50%; and adjusting the matching threshold to 0.3-0.4 when the occlusion is more than 50%.

[0010] Preferably, the illegal intrusion judgment and real-time multi-level warning module comprises illegal intrusion judgment and real-time multi-level warning; and the specific logic of the illegal intrusion judgment is: defining the boundary coordinates of the farmland no-entry area in advance through image segmentation technology; calculating the center point coordinates of the tracked engineering vehicle in real time, and if the coordinates are located within the no-entry area for N consecutive frames, an illegal intrusion event is determined and a warning is triggered, where N is a positive integer greater than or equal to 2; and the real-time multi-level warning comprises: primary warning triggered when the target enters the boundary of the no-entry area within 5 pixels; middle-level warning triggered when the target stays in the no-entry area for more than 3 frames; and high-level warning triggered when the matching degree of the target motion track and the illegal operation mode is more than 70%.

[0011] In the above technical solution, the engineering vehicle detection and tracking method in a farmland under an occlusion environment provided by the application has the following beneficial effects: 1. By introducing the BiFPN and CBAM modules, the feature extraction capability of the model for occlusion and small target engineering vehicles is significantly enhanced, and the detection accuracy (mAP@0.5) is improved to more than 87% in an occlusion scene.

[0012] 2. The improved DeepSort algorithm and the adaptive matching threshold strategy effectively solve the identity maintenance problem after temporary occlusion of the target, and the tracking ID maintenance rate is as high as 96% in the test.

[0013] 3. The entire system is optimized for edge devices, with single-frame processing taking <40ms, meeting the real-time analysis requirements of 25fps and above video streams, and the early warning response time is shortened to within 1 second.

[0014] 4. Through the judgment logic of "continuous multiple frames in the no-entry zone", the accidental errors caused by slight jitter or false detection of the detection box are effectively filtered, and the system false alarm rate is reduced to below 3%, with high reliability.

[0015] 5. The entire system can be deployed on NVIDIA Jetson and other edge computing devices, combined with commercially available surveillance cameras to form a network and implement, with controllable cost, easy to popularize and apply to vast farmland areas.

[0016] A monitoring device using an occluded environment in a farmland engineering vehicle detection and tracking method, comprising: a multispectral image acquisition unit supporting visible light and near-infrared dual-mode acquisition; an edge computing unit configured with a neural network acceleration module; an environmental adaptation unit with an adjustable angle light supplementing device and a protective shell; a communication unit for transmitting monitoring data and warning information.

[0017] Preferably, the multispectral image acquisition unit comprises: a visible light camera for normal monitoring mode; a near-infrared camera for occlusion compensation mode; an automatic switching mechanism that activates the near-infrared mode when the detection confidence drops by more than 10%. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0019] Figure 1 The occluded environment in a farmland engineering vehicle detection and tracking method flowchart provided by the embodiments of the present application. DETAILED DESCRIPTION

[0020] In order to make those skilled in the art better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0021] Please refer to Figure 1The embodiment of the application provides a farmland engineering vehicle detection and tracking method under shielding environment, which comprises the following steps: S1, collecting and labeling farmland engineering vehicle image data under different shielding degrees, constructing a training set, a verification set and a test set, improving and training a YOLOv8 model, constructing a high-precision engineering vehicle detection model, and constructing a data set preparation and model training module; S2, using a high-definition camera to obtain a farmland monitoring video stream in real time, using the improved YOLOv8 model trained in step S1 to process the video frame, outputting an engineering vehicle target detection frame with confidence, and constructing a video stream input and target detection module; S3, inputting the detection frame obtained in step S2 into the improved DeepSort tracking algorithm, predicting the trajectory through Kalman filtering, and using an appearance feature extractor based on EfficientNet-B0 for data association, realizing stable tracking and identity ID maintenance of the engineering vehicle under shielding conditions, and constructing a target tracking and trajectory management module; S4, based on the pre-defined farmland forbidden entry area, analyzing the engineering vehicle motion trajectory obtained in step S3 in real time; when it is determined that the engineering vehicle has illegal intrusion behavior, triggering the early warning mechanism immediately and pushing the alarm information through the remote communication module, and constructing an illegal intrusion judgment and real-time multi-level early warning module.

[0022] It should be noted that during data collection, samples are collected: in corn fields, rice fields and other scenes, videos (resolution 1920x1080, frame rate 25fps) of tractors, harvesters and other engineering vehicles under no shielding, partial shielding (shielding area 30%-50%), and severe shielding (shielding area 50%-70%) are shot, and the cumulative duration is 50 hours; labeling: using LabelImg tool to label the target frame (including class label "engineering vehicle") and mark the shielding degree; dividing the training set (70%), the verification set (20%) and the test set (10%).

[0023] The engineering vehicle detection model introduces a bidirectional feature pyramid network in a YOLOv8 model to realize multi-scale feature fusion, and integrates channel attention and spatial attention double mechanisms at a key feature layer of a backbone network. Further, the basic configuration for training and optimization of the YOLOv8 model includes a PyTorch framework, network improvement, training parameters, and verification indexes; the YOLOv8n is used as a benchmark model based on the PyTorch framework, and the hardware is an NVIDIA RTX 4090 graphics card; meanwhile, the network improvement adds 2 layers of cross-scale feature fusion layers to the Neck part of the PANet, and inserts a CBAM attention module (channel attention + spatial attention); the training parameters are: a learning rate of 0.01, a batch size of 16, 300 iterations, a cosine annealing learning rate scheduling, and a CIoU Loss loss function; and the verification indexes are: stopping training when the mAP@0.5 of the verification set is greater than or equal to 85%, and saving the optimal model weight.

[0024] Specifically, the bidirectional feature pyramid network includes: a top-down feature transmission path for transmitting high-level semantic information to low-level features; a bottom-up feature transmission path for transmitting low-level detail information to high-level features; and a learnable feature weight distribution layer for optimizing the cross-scale feature fusion effect. The channel attention and spatial attention double mechanism includes: a channel attention module generates channel weights through a global pooling operation of a feature map; a spatial attention module generates spatial weights through feature aggregation in the channel dimension; and the two-level attention weights act on the input feature map in turn.

[0025] In the step S3, the improved DeepSort tracking algorithm includes: using a pre-trained EfficientNet-B0 network as an appearance feature extractor, and fine-tuning the appearance feature extractor for engineering vehicle images in a farmland occlusion scene; setting an adaptive matching threshold: when the confidence of a detection box is lower than a preset threshold, automatically reducing the matching threshold of data association to preferentially ensure the continuity of a trajectory after temporary occlusion. Specifically, the DeepSort tracking module deployment includes detector access, feature extraction, and data association. The detector access: uses the trained YOLOv8 model as a detector to output a target box (x, y, w, h) and a confidence (≥0.7); the feature extraction: replaces the original appearance feature extraction network with an EfficientNet-B0 (pre-trained weight initialization), and fine-tunes the feature similarity of the occlusion samples on a test set; and the data association: uses the Hungarian algorithm to match the predicted box (Kalman filter output) and the detection box, and the matching threshold is set to 0.6 (reduced to 0.4 when occluded, to preferentially retain the continuity of the trajectory).

[0026] Specifically, the adaptive matching threshold is as follows: when there is no occlusion, the default matching threshold of 0.6 is maintained; when there is 30%-50% occlusion, the matching threshold is adjusted to 0.4-0.5; when there is more than 50% occlusion, the matching threshold is adjusted to 0.3-0.4.

[0027] The illegal intrusion determination and real-time multi-level early warning module includes illegal intrusion determination and real-time multi-level early warning. The specific logic of the illegal intrusion determination is as follows: the boundary coordinates of the farmland restricted area are predefined through image segmentation technology; the center point coordinates of the tracked engineering vehicle are calculated in real time; if the coordinates are within the restricted area for N consecutive frames, it is determined to be an illegal intrusion event and an early warning is triggered, where N is a positive integer greater than or equal to 2; the real-time multi-level early warning includes: a primary warning, triggered when the target enters within 5 pixels of the restricted area boundary; a secondary warning, triggered when the target stays in the restricted area for more than 3 frames; and a primary warning, triggered when the target's movement trajectory matches the illegal operation mode with a degree of more than 70%. It should be noted that the anomaly detection and early warning system includes farmland area definition, real-time monitoring, and equipment requirements. Farmland area definition: Aerial images of farmland taken by drones are used to segment crop areas as "restricted zones" using Mask R-CNN, and the boundary coordinates are stored. Real-time monitoring: The input video stream is processed frame by frame (each frame takes ≤40ms). An early warning is triggered when the center point coordinates of the engineering vehicle are within the restricted zone for three consecutive frames. Equipment requirements: Edge computing device (NVIDIA Jetson AGX Orin), high-definition camera (with rain and fog penetration capability), and 4G module (for remote push). Specifically, the anomaly early warning response time is 0.8 seconds, meeting the actual needs of farmland monitoring.

[0028] In a preferred embodiment of the present invention, a monitoring device employing a method for detecting and tracking engineering vehicles in farmland under occlusion conditions includes: a multispectral image acquisition unit supporting dual-mode acquisition of visible light and near-infrared light; an edge computing unit equipped with a neural network acceleration module; an environmental adaptation unit with an adjustable-angle supplementary lighting device and a protective shell; and a communication unit for transmitting monitoring data and early warning information. Further, the multispectral image acquisition unit includes: a visible light camera for conventional monitoring mode; a near-infrared camera for occlusion compensation mode; and an automatic switching mechanism that activates the near-infrared mode when the detection confidence level drops by more than 10%.

[0029] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and tracking construction vehicles in farmland under obstructed conditions, characterized in that, Includes the following steps: S1. Collect and label image data of farmland construction vehicles under different degrees of occlusion, construct training set, validation set and test set, improve and train YOLOv8 model, construct high-precision construction vehicle detection model, and construct dataset preparation and model training module. S2. Use a high-definition camera to acquire farmland monitoring video stream in real time, and use the improved YOLOv8 model trained in step S1 to process the video frames, output the engineering vehicle target detection box with confidence, and build a video stream input and target detection module. S3. Input the detection box obtained in step S2 into the improved DeepSort tracking algorithm, predict the trajectory through Kalman filtering, and use the appearance feature extractor based on EfficientNet-B0 for data association to achieve stable tracking and identity ID maintenance of the engineering vehicle under occlusion conditions, and build a target tracking and trajectory management module. S4. Based on the predefined farmland restricted area, analyze the movement trajectory of the engineering vehicle obtained in step S3 in real time; when it is determined that the engineering vehicle has committed an illegal intrusion, immediately trigger the early warning mechanism and push the alarm information through the remote communication module to build an illegal intrusion judgment and real-time multi-level early warning module.

2. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 1, characterized in that, The engineering vehicle detection model introduces a bidirectional feature pyramid network into the YOLOv8 model to achieve multi-scale feature fusion; and integrates channel attention and spatial attention mechanisms in the key feature layer of the backbone network.

3. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 2, characterized in that, The bidirectional feature pyramid network includes: a top-down feature transfer path that transfers high-level semantic information to low-level features; a bottom-up feature transfer path that transfers low-level detail information to high-level features; and a learnable feature weight allocation layer for optimizing cross-scale feature fusion.

4. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 2, characterized in that, The dual mechanism of channel attention and spatial attention includes: the channel attention module generates channel weights through global pooling of feature maps; the spatial attention module generates spatial weights through feature aggregation in the channel dimension; and the two levels of attention weights are applied to the input feature map in sequence.

5. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 1, characterized in that, The improved DeepSort tracking algorithm in step S3 includes: using a pre-trained EfficientNet-B0 network as an appearance feature extractor and fine-tuning it for engineering vehicle images in farmland occlusion scenarios; setting an adaptive matching threshold: when the confidence of the detection box is lower than a preset threshold, the matching threshold of data association is automatically reduced to prioritize ensuring the continuity of the trajectory after a brief occlusion.

6. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 5, characterized in that, The adaptive matching thresholds are as follows: when there is no occlusion, the default matching threshold of 0.6 is maintained; when there is 30%-50% occlusion, the matching threshold is adjusted to 0.4-0.5; when there is more than 50% occlusion, the matching threshold is adjusted to 0.3-0.

4.

7. The method for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 1, characterized in that, The illegal intrusion detection and real-time multi-level early warning module includes illegal intrusion detection and real-time multi-level early warning; The specific logic for determining illegal intrusion is as follows: the boundary coordinates of the restricted area in farmland are predefined using image segmentation technology; the center point coordinates of the tracked engineering vehicle are calculated in real time; if the coordinates are within the restricted area for N consecutive frames, it is determined to be an illegal intrusion event and an early warning is triggered, where N is a positive integer greater than or equal to 2. The real-time multi-level early warning includes: a primary warning, triggered when the target enters within 5 pixels of the restricted area boundary; a secondary warning, triggered when the target remains in the restricted area for more than 3 frames; and a primary warning, triggered when the target's movement trajectory matches the illegal operation mode by more than 70%.

8. A monitoring device employing the method for detecting and tracking construction vehicles in farmland under obstructed conditions as described in any one of claims 1-7, characterized in that, include: A multispectral image acquisition unit supporting dual-mode acquisition of visible light and near-infrared light; an edge computing unit equipped with a neural network acceleration module; and an environmental adaptation unit with an adjustable-angle supplementary lighting device and a protective shell. A communication unit used to transmit monitoring data and early warning information.

9. The monitoring device for detecting and tracking construction vehicles in farmland under obstructed conditions according to claim 8, characterized in that, The multispectral image acquisition unit includes: a visible light camera for conventional monitoring mode; a near-infrared camera for occlusion compensation mode; and an automatic switching mechanism that activates near-infrared mode when the detection confidence level drops by more than 10%.

Images