A hilly and mountainous track obstacle identification and grading control method and system based on improved YOLOv8

Abstract

The application provides a hilly and mountainous track obstacle identification and grading control method and system based on improved YOLOv8, utilizes an industrial camera to collect a video stream in front of a track and deliver to a master control module, the master control module uses an improved YOLOv8 model to infer and identify obstacles, and further controls the switching management of the transportation equipment in four states of normal operation, temporary avoidance, recovery confirmation and fault locking; the improved YOLOv8 model includes StarNet backbone network, CAA attention module, FPN-PAN neck network and detection head. The application solves the problems that the existing agricultural implement track transportation equipment relies on manual observation, cannot identify obstacles independently and respond, and the existing visual recognition algorithm has poor detection accuracy and real-time performance for obstacles with obvious length-width ratio and small size targets in complex mountainous environments.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and agricultural automation technology, and in particular relates to a method and system for obstacle recognition and hierarchical control in hilly and mountainous terrain based on an improved YOLOv8. Background Technology

[0002] Mountain rail transport vehicles are key equipment for transporting materials in hilly and mountainous areas. However, existing agricultural machinery rail transport equipment generally relies on manual observation or on-board operation, which is not only inefficient and costly but also poses significant safety hazards. Existing technology (Jia Chuang, Li Jianian, Hong Tiansheng, et al. Design and Experiment of Ultrasonic Obstacle Avoidance System for Monorail Transport Vehicle in Mountain Orchards [J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(Z2):69-74.) uses ultrasonic sensors to achieve obstacle perception and automatic stopping. However, this solution suffers from problems such as short detection distance, low angular resolution, and inability to identify obstacle types, making it difficult to meet the operational needs of complex mountainous environments. Furthermore, although some studies have attempted to introduce machine vision technology to achieve autonomous recognition, the use of custom-designed simple CNN networks lacks an attention mechanism and is easily interfered with in complex backgrounds such as vegetation obstruction and changing lighting. Therefore, this invention designs a method and system for obstacle recognition and hierarchical control of mountain rail transport vehicles based on an improved YOLOv8. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method and system for obstacle recognition and hierarchical control in hilly and mountainous terrain based on an improved YOLOv8. This solves the problems of existing agricultural machinery track transport equipment relying on manual observation and being unable to autonomously identify obstacles and respond, as well as the problems of existing visual recognition algorithms having insufficient accuracy and poor real-time performance in complex mountainous environments for detecting obstacles with significant aspect ratios and small-sized targets.

[0004] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0005] A method for obstacle identification and hierarchical control in hilly and mountainous terrain based on improved YOLOv8 includes the following process:

[0006] Step 1: The transport equipment transports agricultural machinery on the track, and the industrial camera collects image data in front of the track and transmits it to the main control module in real time;

[0007] Step 2: The main control module inputs the preprocessed image into the improved YOLOv8 model to perform forward propagation inference. The improved YOLOv8 model consists of four parts: StarNet backbone network, CAA attention module, FPN-PAN neck network, and detection head. The input image first passes through the StarNet backbone network for lightweight feature extraction, generating feature maps at three scales. These feature maps are then input into the CAA attention module for weight calibration, enhancing the features of the track obstacle and suppressing interference from complex backgrounds. The calibrated feature maps are then fed into the FPN-PAN neck network for multi-scale feature fusion, resulting in a fused feature map at three scales. This fused feature map is then input into the detection head, which finally outputs the obstacle's category label, confidence score, and bounding box coordinates.

[0008] Step 3: The main control module performs hierarchical control of the transportation equipment based on the inference results output by the improved YOLOv8 model;

[0009] Step 3.1: The track access area generation unit generates the track access area ROI based on the industrial camera calibration parameters;

[0010] Step 3.2: The validity determination unit calculates the intersection-over-union ratio (IoU) of the obstacle bounding box and the ROI. ,like If it is an obstacle outside the track, then it is determined to be an obstacle outside the track; if If so, it is determined to be a valid track obstacle;

[0011] Step 3.3: The size estimation unit further estimates the lateral width and height of the obstacle in the real world based on the bounding box pixel size and industrial camera calibration parameters, and compares them with the track clearance threshold and wheel radius threshold to determine whether it is a passable object or an impassable static obstacle.

[0012] Step 3.4: The status control unit performs the switching and management of four states: normal operation, temporary avoidance, recovery confirmation, and fault lockout.

[0013] Furthermore, in step 2, the StarNet backbone network adopts a four-stage layered architecture, with each stage consisting of multiple stacked star operation modules. The data processing process of the star operation modules is as follows:

[0014] Step 2.1.1: Input feature map Send in Depthwise separable convolutional layers are used for spatial feature extraction to obtain the output feature map of the depthwise separable convolution. ;

[0015] Step 2.1.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Two are fed in parallel The convolutional layer expands the number of channels to four times the original number, obtaining output feature maps for two branches. , ;

[0016] Step 2.1.3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation and Element-wise multiplication is performed to complete the star operation, resulting in the output feature map of the star operation. ;

[0017] Step 2.1.4: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation The batch normalization layer performs normalization processing to obtain the batch normalized feature map. ;

[0018] Step 2.1.5: Through Convolutional layers restore the number of channels to their original dimensions, obtaining feature maps with restored channels. ;

[0019] Step 2.1.6: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Passing by again After depthwise separable convolution, it is compared with the original input feature map. The residues are added together to form a residual connection, which yields the output feature map of the star operation module.

[0020] Step 2.1.7: Stack multiple star operation modules as described above, and then stack the first... The output of the star operation module is used as the first Input to the individual star operation module;

[0021] Step 2.1.8: By stacking multiple star operation modules and using convolutional layers with a stride of 2 for downsampling, the StarNet backbone network finally outputs three scale feature maps: shallow feature map, medium feature map, and deep feature map.

[0022] Furthermore, in step 2, a dynamic strip convolution kernel size adjustment mechanism based on local texture complexity is introduced into the CAA attention module, enabling the receptive field of the strip convolution to adaptively adjust according to the content difference between the obstacle and the background. The specific data processing procedure is as follows:

[0023] Step 2.2.1: For the three scale feature maps input to the StarNet backbone network, firstly through... The average pooling layer captures local region features, resulting in a feature map after average pooling. ;

[0024] Step 2.2.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation pass The convolutional layer performs feature transformation to obtain the convolutional feature map. ;

[0025] Step 2.2.3: To perceive the content differences at different spatial locations in the feature map, calculate the local texture complexity score for each location; first, ... pass Convolutional compression into single-channel feature maps ; then on Each spatial location Take a point centered on that location. Local window, Given a preset local window side length, calculate the variance of all pixel values ​​within that window as the local texture complexity score for that location. :

[0026]

[0027]

[0028] In the formula, The coordinates of the center pixel for the current computational complexity score; These are the horizontal and vertical offsets relative to the center pixel within the local window, respectively, and their values ​​range from [ ]. , ]; For Centered The average value of all pixels within the window; Indicates rounding down;

[0029] The complexity scores of all spatial locations constitute a complexity graph. ;

[0030] Step 2.2.4: Based on the complexity graph For each spatial location, a set of attention weights is generated to fuse bar convolutional kernels of multiple scales; first, the following settings are defined. Candidate bar convolution kernels of different sizes, i.e., candidate convolution kernels, are given different receptive fields to capture short-range, medium-range, and long-range spatial contexts, respectively; the complexity graph is then used to... Each spatial location Complexity score Input a weight generator consisting of two fully connected layers, and output the weights corresponding to that position after Softmax normalization. Attention weights for each candidate convolutional kernel:

[0031]

[0032] In the formula, Spatial location place dimensional attention weight vector, Indicates spatial location Assigned to the The weights of each candidate convolutional kernel, ; and These are two fully connected layers; ReLU is the activation function; Softmax is the normalization function.

[0033] Step 2.2.5: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Enter them separately A number of parallel horizontal and vertical depth-separable strip convolutions, the first... The convolution uses the method defined in step 2.2.4. One candidate convolutional kernel;

[0034] Using the attention weights generated in step 2.2.4, for each spatial location of The outputs corresponding to each candidate convolutional kernel are weighted and fused to obtain the strip convolutional features of the adaptive receptive field; after weighted fusion of all spatial locations, a complete horizontal fusion feature map is formed. Feature maps fused with vertical direction ;

[0035] Step 2.2.6: Fuse the strip convolution outputs from the two directions, and then... Convolutional layers and the sigmoid activation function generate attention weight maps ;

[0036] Step 2.2.7: Calculate the attention weight map Element-wise multiplication with the three scale feature maps input from the StarNet backbone network yields refined feature maps at three scales along the channel dimension.

[0037] Furthermore, in step 2.2.5, the following will be performed: Enter them separately A number of parallel horizontal and vertical depth-separable strip convolutions, the first... The convolution uses the method defined in step 2.2.4. One candidate convolutional kernel:

[0038]

[0039]

[0040] In the formula, , They are respectively using the first The horizontal and vertical strip convolution outputs of the candidate convolution kernels; Indicates the kernel size as Depth-separable convolution, Indicates the kernel size as Depth-separable convolution;

[0041] The strip convolutional features of the adaptive receptive field are as follows:

[0042]

[0043]

[0044] In the formula, The fused horizontal strip convolution output feature map is located in space. eigenvalues ​​at that location The fused vertical strip convolution output feature map is located in space. Eigenvalues ​​at; To adopt the first The horizontal strip convolution of the candidate convolution kernels in spatial location Output feature value at, To adopt the first The vertical strip convolution of each candidate convolution kernel in spatial location The output feature value at that location.

[0045] Furthermore, in step 2, the improved YOLOv8 model is trained offline before deployment. The training dataset covers common obstacle types including falling rocks, fallen branches, branches crossing the track, and small animals. During training, data augmentation strategies including Mosaic enhancement, geometric transformation, color adjustment, and random flipping are used to improve the model's generalization ability. The bounding box regression loss function adopts a self-improved Wise-MPDIoU loss function, the calculation formula of which is:

[0046]

[0047] In the formula, , These are the prediction boxes. The coordinates of the corner points; , The real frame number is 1. The coordinates of the corner points , To predict the width and height of the bounding box, As a penalty for aspect ratio consistency, It is a dynamic non-monotonic focusing factor. , For balance coefficient, It is a very small constant.

[0048] Furthermore, in step 3, if the validity determination unit determines that it is an obstacle outside the track, the transportation equipment continues to operate normally; if the validity determination unit determines that it is a valid track obstacle, the size estimation unit makes a further determination; when it is further determined to be a passable object, the status control unit controls the transportation equipment to switch to a temporary avoidance state; when it is further determined to be a non-passable static obstacle, the status control unit controls the transportation equipment to switch to a fault lock state.

[0049] Furthermore, when the transport equipment is in a temporary obstacle avoidance state, the drive control module outputs a temporary braking signal, causing the transport equipment to decelerate and stop; the continuous frame confirmation unit sets a continuous obstacle-free frame counter. The initial value is 0. The industrial camera 102 continuously collects data. For each frame, if no obstacle is detected within the track travel area and the detection confidence is less than 0.5, then... ,otherwise ;when When the device is in the recovery confirmation state, the drive control module outputs a recovery operation command, the transportation equipment starts running, and then returns to normal operation.

[0050] Furthermore, when the transport equipment is in a fault-locked state, the drive control module outputs an emergency braking signal and blocks all start signals; it sends a first-level alarm message to the host computer control system, including the obstacle type, size, location coordinates, and timestamp; after manual obstacle removal is completed, the host computer control system issues a remote reset command, and the main control module releases the lock and restores the system to standby mode after receiving the reset command.

[0051] A system for implementing the above-mentioned obstacle recognition and hierarchical control method for hilly and mountainous tracks includes an image acquisition module, a main control module, a protective structure, and a communication module. The image acquisition module includes an industrial camera for acquiring image data in front of the track and transmitting it to the main control module for analysis and processing, identifying obstacle categories, and controlling the transportation equipment to switch states. The protective structure provides physical protection for the core components of the system. The communication module transmits the recognition results from the main control module to the outside. The main control module integrates a drive control module and the following functional units: a track passage area generation unit, an effectiveness determination unit, a size estimation unit, a state control unit, a continuous frame confirmation unit, and an embedded AI computing unit.

[0052] Furthermore, the image acquisition module is fixed to the transport equipment control box via a mounting bracket, and the main control module is fixed inside the transport equipment control box. The protective structure is made of aluminum alloy and is installed on the transport equipment control box, enclosing the image acquisition module. The mounting bracket includes a base, an adjusting arm, and a locking mechanism. The base is fixed to the control box. The adjusting arm is connected to the vertical rod of the base via a universal joint, achieving a spatial angle adjustment of ±30°. The locking mechanism acts on the rotating joint of the universal joint through a wing nut, and by tightening the wing nut, a locking force is applied to the rotating mating surface of the universal joint, thereby locking the adjusting arm relative to the vertical rod. The industrial camera is mounted on the adjusting arm.

[0053] The present invention has the following beneficial effects:

[0054] Through collaborative hardware and software design, this invention comprehensively improves the safety and operational continuity of agricultural machinery rail transport in hilly and mountainous areas. Based on an improved YOLOv8 automatic identification and hierarchical control method, it effectively reduces reliance on manual observation. For complex mountainous scenarios, a lightweight StarNet backbone is adopted, which achieves implicit high-dimensional feature mapping in a compact network through star operations, significantly reducing the number of parameters and computational cost while enhancing feature representation capabilities. A CAA attention module is introduced, which enhances the feature extraction capability for obstacles with significant aspect ratios and small sizes through strip convolution, and suppresses complex background interference such as vegetation occlusion and lighting changes. Furthermore, the strip convolution mechanism of the CAA attention module is improved by introducing a dynamic convolution kernel size adjustment mechanism based on local texture complexity. By calculating the local feature variance at each spatial location, the difference between the obstacle and the background is perceived, so that the obstacle region automatically obtains a larger receptive field to capture sufficient spatial context, while the background region automatically obtains a smaller receptive field to avoid irrelevant noise interference. Thus, the feature extraction capability and background suppression capability of multi-scale obstacles in complex backgrounds are significantly improved with almost no increase in computational cost. The Wise-MPDIoU loss function is adopted to solve the localization failure problem when the aspect ratio is the same through multi-point distance calculation, thereby significantly improving recognition accuracy and robustness. The system implements tiered control based on obstacle type. Static obstacles trigger emergency shutdown and alarms, while dynamic obstacles trigger temporary shutdown and automatic resumption of operation, avoiding prolonged downtime caused by brief animal disturbances. Furthermore, the system's protective and shock-absorbing structure provides a stable and reliable hardware operating environment for the algorithm. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the overall structure of the hilly and mountainous track obstacle recognition and hierarchical control system described in this invention;

[0056] Figure 2 This is a schematic diagram of the mounting bracket structure described in this invention;

[0057] Figure 3 This is a schematic diagram of the installation of the image acquisition module described in this invention;

[0058] Figure 4 This is a schematic diagram of the installation of the protective structure described in this invention;

[0059] Figure 5 This is a schematic diagram of the installation of the main control module described in this invention;

[0060] Figure 6 This is a schematic diagram of the overall architecture of the improved YOLOv8 model described in this invention;

[0061] Figure 7 This is a schematic diagram of the data processing process of the star operation module described in this invention;

[0062] Figure 8 This is a schematic diagram of the data processing process of the CAA attention module described in this invention;

[0063] Figure 9 This is a schematic diagram of the hierarchical control process of the present invention;

[0064] In the diagram: 101-Image acquisition module; 102-Industrial camera; 103-Strip LED lighting source; 104-Camera mounting plate; 105-Hex bolt; 106-Light source bracket; 107-Elongated hole; 108-Arc groove; 201-Main control module; 301-Protective structure; 302-Front cover; 303-Main housing; 304-Sealing bolt; 305-Silicone sealing ring; 401-Communication module; 501-Mounting bracket; 502-Base; 503-Adjusting arm; 504-Locking mechanism; 505-Stainless steel bolt; 506-Universal joint; 507-Wing nut; 601-Anti-vibration mounting base; 602-Rubber shock-absorbing pad; 603-Metal bracket; 604-Bolt; 701-Control box. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0066] like Figures 1 to 5 As shown, the hilly and mountainous track obstacle recognition and classification control system based on the improved YOLOv8 described in this invention is installed on agricultural machinery track transportation equipment and includes an image acquisition module 101, a main control module 201, a protective structure 301, and a communication module 401. The image acquisition module 101 is used to acquire images of the track ahead; the main control module 201 is used to run the improved YOLOv8 model to recognize and process the images acquired by the image acquisition module 101; the protective structure 301 is used to provide physical protection for the core components of the system in harsh mountainous environments; and the communication module 401 is used to transmit the recognition results of the main control module 201 to the outside.

[0067] like Figure 1 , 2 As shown, the image acquisition module 101 is fixed to the central position of the control box 701 of the agricultural machinery track transport equipment via a mounting bracket 501. The mounting bracket 501 adopts a spatial multi-degree-of-freedom adjustment design, including a base 502, an adjusting arm 503, and a locking mechanism 504. The base 502 is fixed to the control box 701 in a cross-symmetrical distribution using stainless steel bolts 505. The adjusting arm 503 is connected to the vertical rod of the base 502 via a universal joint 506, enabling a spatial angle adjustment of ±30°. The locking mechanism 504 includes a wing nut 507, which acts on the rotating joint of the universal joint 506. By tightening the wing nut 507, a locking force is applied to the rotating mating surface of the universal joint 506, thereby locking the adjusting arm 503 relative to the vertical rod and ensuring that no positional shift occurs under the vibration environment of track transport.

[0068] like Figure 2 , 3 As shown, the core sensing unit of the image acquisition module 101 includes an industrial camera 102 and a strip LED lighting source 103. The industrial camera 102 is connected to the adjusting arm 503 via a camera mounting plate 104 with elongated holes 107. The camera mounting plate 104 and the adjusting arm 503 are fixed together with hexagonal bolts 105. The design of the elongated holes 107 allows for millimeter-level precision in adjusting the front and rear positions of the industrial camera 102 along the optical axis. The strip LED lighting source 103 is symmetrically arranged on the adjusting arms 503 on both sides of the industrial camera 102 via light source brackets 106 with arc-shaped grooves 108. Its illumination angle can be infinitely adjusted within a range of ±45°. It employs a low-angle sweeping illumination method to provide stable and uniform illumination for image acquisition at night or in low-light conditions, effectively overcoming shadow interference.

[0069] like Figure 1 , 5 As shown, the main control module 201 is fixed inside the control box 701 of the agricultural machinery track transport equipment via a shock-absorbing mounting base 601. The shock-absorbing mounting base 601 includes an oil-resistant rubber shock-absorbing pad 602 and a metal bracket 603. The shock-absorbing mounting base 601 is connected to the bottom of the control box 701 via bolts 604, forming a two-stage shock absorption system, which can effectively absorb and isolate high-frequency vibrations and instantaneous impacts generated during track transport operations. The main control module 201 integrates a drive control module and the following functional units: a track passage area generation unit, an effectiveness determination unit, a size estimation unit, a status control unit, a continuous frame confirmation unit, and an embedded AI calculation unit. Among them, the drive control module is used to directly control the temporary braking, emergency braking, start-up operation, or shielding of the start-up signal of the transport equipment according to the output instructions of the status control unit.

[0070] like Figure 1 , 4 As shown, the protective structure 301 is a high-protection-level sealed shell structure made of aluminum alloy. It is installed on the control box 701 of the agricultural machinery track transport equipment and encloses the image acquisition module 101, providing comprehensive dust and splash protection for the image acquisition module 101. The protective structure 301 includes a front cover 302 and a main shell 303. The front cover 302 and the main shell 303 are connected by sealing bolts 304, and a silicone sealing ring 305 is provided at the joint surface. A tempered glass transparent window with anti-glare treatment is directly embedded in the front cover 302, with a light transmittance of 92%, achieving dust and splash protection without affecting the image acquisition module 101's image acquisition.

[0071] The obstacle identification and hierarchical control method for hilly and mountainous tracks based on the improved YOLOv8 described in this invention includes the following process:

[0072] Step 1: System initialization and image acquisition;

[0073] After the system is powered on, the agricultural implement track transport equipment moves on the track to transport agricultural implements; the main control module 201 starts up and loads the improved YOLOv8 model into the memory of its embedded AI computing unit; the image acquisition module 101 is activated simultaneously, and the industrial camera 102 begins to acquire video streams in front of the track at a resolution of 1920x1080 and a frame rate of 30fps; the strip LED lighting source 103 is turned on to provide stable and uniform auxiliary lighting for image acquisition; the industrial camera 102 acquires the raw image data.

[0074] Step 2: Improve the reasoning and recognition of the YOLOv8 model;

[0075] The main control module 201 preprocesses the received image, including scaling the size to 640×640 and normalizing the pixel values; then, the preprocessed image tensor is input into the improved YOLOv8 model to perform forward propagation inference.

[0076] The overall architecture of the improved YOLOv8 model is as follows: Figure 6As shown, the system comprises four parts: the StarNet backbone, the CAA attention module (located at the end of the backbone), the FPN-PAN neck network, and the head. Its forward propagation inference process is as follows: The input image first undergoes lightweight feature extraction through the StarNet backbone, generating feature maps at three scales; these feature maps are then input into the CAA attention module for weight calibration, enhancing the features of the orbital obstacles and suppressing interference from complex backgrounds; the calibrated feature maps are then fed into the FPN-PAN neck network for multi-scale feature fusion; finally, the fused feature maps are input into the head, outputting the obstacle's category label, confidence score, and bounding box coordinates. The detailed processing steps are as follows:

[0077] Step 2.1: StarNet backbone network data processing;

[0078] To meet the real-time requirements of the embedded platform for rail transportation equipment, the backbone network of the original YOLOv8 model was replaced with StarNet. StarNet adopts a four-stage hierarchical architecture, with each stage consisting of multiple stacked star operation modules. Downsampling is performed through convolutional layers with a stride of 2, and the network width doubles with each stage, while the channel expansion factor remains fixed at 4. To adapt to the YOLOv8 detection head, feature maps downsampled at 8x, 16x, and 32x scales are selected as outputs for detecting small, medium, and large obstacles, respectively. The core of StarNet is the star operation module (Star Block), and its data processing process is as follows: Figure 7 As shown, given the input feature map , Represents the real number field. This represents the number of channels in the input feature map. Indicates the height of the input feature map. The width of the input feature map is represented by the star operation module, and the calculation process is as follows:

[0079] Step 2.1.1: Depthwise separable convolution;

[0080] Input feature map Send in Depthwise separable convolutional layers for spatial feature extraction:

[0081]

[0082] In the formula, The output feature map of depthwise separable convolution. Indicates the kernel size as Depth-separable convolution operations, This is the input feature map.

[0083] Step 2.1.2: Channel expansion and two-branch transformation;

[0084] Will Two are fed in parallel Convolutional layers expand the number of channels to four times the original number:

[0085]

[0086]

[0087] In the formula, , The output feature maps of the two branches, , for The weight matrix of the convolutional layer, , This is a bias term.

[0088] Step 2.1.3: Star operation;

[0089] Will and Element-wise multiplication to complete the star operation:

[0090]

[0091] In the formula, The output feature map of the star operation. This indicates element-wise multiplication.

[0092] Step 2.1.4: Batch normalization;

[0093] Feature map of star operation output The batch normalization layer is normalized to stabilize the training process and prevent gradient explosion.

[0094]

[0095] In the formula, This is the batch normalized feature map; This indicates batch normalization operation.

[0096] Step 2.1.5: Channel restoration;

[0097] pass Convolutional layers restore the number of channels to the original dimension:

[0098]

[0099] In the formula, This is the feature map after channel recovery. for The weight matrix of the convolutional layer, Indicates matrix transpose. This is a bias term.

[0100] Step 2.1.6: Residual connection;

[0101] Passing by again After depthwise separable convolution, it is compared with the original input feature map. Add them together to form a residual connection:

[0102]

[0103] In the formula, This is the output feature map of the star operation module. Indicates the kernel size as Depth-separable convolution operations.

[0104] in, Approved Convolutional layers restore the number of channels to the input feature map. With the same original dimensions, depthwise separable convolutions do not change the number of channels in the feature map, therefore and The dimensions are consistent.

[0105] Step 2.1.7: Module stacking;

[0106] Stack multiple star operation modules as described above, the first... The output of the star operation module is used as the first Input to the individual star operation module:

[0107]

[0108] In the formula, For the first The output feature map of the individual star operation module This indicates the star operation module. For the first The output feature map of the star operation module (i.e., the first star operation module) (Input feature map of the individual star operation module).

[0109] Step 2.1.8: Multi-scale feature output;

[0110] By stacking multiple star operation modules and using convolutional layers with a stride of 2 for downsampling, the StarNet backbone network finally outputs feature maps at three scales:

[0111]

[0112] In the formula, This is a shallow feature map. This represents the number of channels in the shallow feature map. It is a spatial dimension responsible for detecting small obstacles; This is a mid-level feature map. This represents the number of channels in the mid-layer feature map. It has spatial dimensions and is responsible for detecting medium-sized obstacles; For deep feature maps, This represents the number of channels in the deep feature map. It is a spatial dimension responsible for detecting large obstacles.

[0113] Step 2.2: Embedding CAA attention module data processing;

[0114] like Figure 8 As shown, the feature maps at three scales output by the StarNet backbone network are... , , The inputs are respectively processed by the CAA attention module embedded at the end of the StarNet backbone network; the CAA attention module captures remote context information through average pooling and bar convolution. This invention further improves the CAA attention module by introducing a dynamic bar convolution kernel size adjustment mechanism based on local texture complexity, enabling the receptive field of the bar convolution to adaptively adjust according to the content difference between obstacles and the background. The calculation process is as follows:

[0115] Step 2.2.1: Average pooling;

[0116] For the input scale feature map , ( include , , ,for , ,for , ,for , ), Represents the real number field. The number of channels in the scale feature map. Indicates the height of the scale feature map. The width of the scale feature map is first represented by... Average pooling layers acquire local region features:

[0117]

[0118] In the formula, This is the feature map after average pooling. Indicates the kernel size as An average pooling operation with a step size of 1 and padding of 3 is performed, which maintains the same spatial size of the output feature map as the input.

[0119] Step 2.2.2: Feature transformation;

[0120] The feature map after average pooling is then processed... Convolutional layers perform feature transformations:

[0121]

[0122] In the formula, The feature map after convolution. express Convolution operation.

[0123] Step 2.2.3: Local texture complexity estimation;

[0124] To perceive the content differences at different spatial locations in the feature map, this step calculates the local texture complexity score for each location; firstly, the feature map after feature transformation is... pass Convolutional compression into a single-channel feature map:

[0125]

[0126] In the formula, This is the compressed single-channel feature map. express Convolution operation.

[0127] Subsequently, Each spatial location Take a point centered on that location. A partial window, in which The preset local window side length (in this example, it is taken as...) ), calculate the variance of all pixel values ​​within the window, and use it as the local texture complexity score for that location. :

[0128]

[0129]

[0130] In the formula, The coordinates of the center pixel for the current computational complexity score; These are the horizontal and vertical offsets relative to the center pixel within the local window, respectively, and their values ​​range from [ ]. , ]; For Centered The average value of all pixels within the window; This indicates rounding down to the nearest integer.

[0131] The complexity scores of all spatial locations constitute a complexity graph. , Obstacle regions, due to their rich texture and distinct edges, typically exhibit higher local variance; while uniform background regions exhibit lower local variance.

[0132] Step 2.2.4: Dynamic kernel size weight generation;

[0133] Based on complexity graph For each spatial location, a set of attention weights is generated to fuse bar convolutional kernels of multiple scales. First, define... Candidate strip convolution kernels of different sizes (i.e., candidate convolution kernels) are selected to have different receptive fields (in this embodiment, we take...). The candidate kernel sizes are respectively Each of these is responsible for capturing short-range, medium-range, and long-range spatial contexts, respectively.

[0134] Complexity graph Each spatial location Complexity score Input a weight generator consisting of two fully connected layers, and output the weights at that position after Softmax normalization. Attention weights for each candidate convolutional kernel:

[0135]

[0136] In the formula, Spatial location place The attention weight vector satisfies , Indicates spatial location Assigned to the The weights of each candidate convolutional kernel, ; and These are two fully connected layers; ReLU is the activation function; and Softmax is the normalization function.

[0137] This mechanism automatically grants a larger effective receptive field to obstacle regions with high complexity and a smaller effective receptive field to background regions with low complexity, thereby suppressing background noise interference while capturing long-range context of obstacles.

[0138] Step 2.2.5: Multi-scale strip convolution and spatial adaptive fusion;

[0139] Feature map after feature transformation Enter them separately A number of parallel horizontal and vertical depth-separable strip convolutions, the first... The convolution uses the method defined in step 2.2.4. One candidate convolutional kernel:

[0140]

[0141]

[0142] In the formula, , They are respectively using the first The horizontal and vertical strip convolution outputs of the candidate convolution kernels; Indicates the kernel size as Depth-separable convolution, Indicates the kernel size as Depth-separable convolutions.

[0143] Using the attention weights generated in step 2.2.4, for each spatial location of The outputs corresponding to the candidate convolutional kernels are weighted and fused to obtain the strip convolutional features with an adaptive receptive field:

[0144]

[0145]

[0146] In the formula, The fused horizontal strip convolution output feature map is located in space. eigenvalues ​​at that location The fused vertical strip convolution output feature map is located in space. Eigenvalues ​​at; To adopt the first The horizontal strip convolution of the candidate convolution kernels in spatial location Output feature value at, To adopt the first The vertical strip convolution of each candidate convolution kernel in spatial location The output feature value at that location.

[0147] After the above weighted fusion, all spatial locations constitute a complete horizontal fused feature map. Feature maps fused with vertical direction .

[0148] Step 2.2.6: Feature fusion and attention generation;

[0149] The outputs of the two-directional strip convolutions are fused together, and then... Convolutional layers and the sigmoid activation function generate attention weight maps:

[0150]

[0151]

[0152] In the formula, The output of the fused feature map is generated by the two-way strip convolution. For the generated attention weight map, It is the Sigmoid activation function. express Convolution operation; attention weight map Spatial dimensions and input scale feature map Same, channel number is 1.

[0153] Step 2.2.7: Feature Enhancement;

[0154] Attention weight map scale feature map of input Element-wise multiplication yields a refined feature map along the channel dimension. :

[0155]

[0156] In the formula, To improve the output feature map of the CAA attention module, including , , ;

[0157] The refined feature maps at the three scales output by the CAA attention module are denoted as follows:

[0158]

[0159] In the formula, This is a shallow refined feature map. This is a mid-level refined feature map. This is a deeply refined feature map.

[0160] Step 2.3: FPN-PAN neck network processing;

[0161] Refined feature maps at three scales output by the CAA attention module , , The input is fed into the FPN-PAN neck network for multi-scale feature fusion, resulting in fused feature maps at three scales, denoted as follows: , , .

[0162] Step 2.4: Detection head prediction;

[0163] The fused feature map of the three scales output by the FPN-PAN neck network , , The data is input into the corresponding detection heads. Each detection head consists of a classification branch and a regression branch. The classification branch outputs the probability distribution of the obstacle category, and the regression branch outputs the bounding box coordinates. After the prediction results at all scales are merged, redundant boxes are removed using non-maximum suppression, and the final output is the obstacle's category label, confidence score, and bounding box coordinates.

[0164] The improved YOLOv8 model described above requires offline training before deployment. Training is conducted on a track obstacle dataset containing 15,000 labeled images, covering common obstacle types such as falling rocks, fallen branches, branches crossing the track, and small animals. During training, data augmentation strategies such as Mosaic enhancement, geometric transformation, color adjustment, and random flipping are employed to improve the model's generalization ability.

[0165] The bounding box regression loss function adopts the self-improved Wise-MPDIoU loss function, and its calculation formula is as follows:

[0166]

[0167] In the formula, , These are the prediction boxes. The coordinates of the corner points; , The real frame number is 1. The coordinates of the corner points , To predict the width and height of the bounding box, As a penalty for aspect ratio consistency, It is a dynamic non-monotonic focusing factor. , For balance coefficient, It is a very small constant (taken in this embodiment). ), used to avoid the denominator being zero.

[0168] This loss function solves the failure problem of traditional IoU under the same aspect ratio by calculating the distance between multiple points, and at the same time uses a dynamic focusing mechanism to reduce the harmful gradient of low-quality samples.

[0169] Step 3: Hierarchical control and execution;

[0170] After the improved YOLOv8 model inference is completed, the main control module 201 parses the model output to obtain the obstacle category labels, confidence scores, and bounding box coordinates. Based on this inference result, hierarchical control is performed, referring to... Figure 9 As shown, the specific process is as follows:

[0171] Step 3.1: The track access area generation unit generates the track access area ROI based on the industrial camera calibration parameters.

[0172] Step 3.2: The validity determination unit calculates the intersection-union ratio (IU) of the obstacle bounding box and the ROI. ),like If it is an obstacle outside the track, it is ignored; if If it is, then it is determined to be a valid track obstacle.

[0173] Step 3.3: The size estimation unit further estimates the lateral width and height of the obstacle in the real world based on the bounding box pixel size and industrial camera calibration parameters, and compares them with the track clearance threshold and wheel radius threshold to determine whether it is a passable object or an impassable static obstacle.

[0174] Step 3.4: The status control unit performs switching and management of four states: normal operation, temporary avoidance, recovery confirmation, and fault lockout, as detailed below:

[0175] (1) When in normal operation, if the validity determination unit determines that it is an obstacle outside the track, the transport equipment continues to operate; if the validity determination unit determines that it is a valid track obstacle, the size estimation unit will make a further determination. When it is further determined that it is a passable object (e.g., a small animal), the status control unit controls the transport equipment to switch to the temporary avoidance state. When it is further determined that it is an impassable static obstacle (e.g., the size of a falling rock is greater than the track gap, or the height of a fallen tree branch is greater than the wheel radius), the status control unit controls the transport equipment to switch to the fault lock state.

[0176] (2) When in a temporary avoidance state, the drive control module outputs a temporary braking signal (braking deceleration is 0.2g to 0.3g, where g represents gravitational acceleration), and the transport equipment decelerates and stops; the continuous frame confirmation unit sets a continuous obstacle-free frame counter. The initial value is 0. The image acquisition module 101 continuously acquires images at a frame rate of no less than 15 fps. For each frame, if no dynamic obstacle is detected within the ROI (Region of Interest) and the detection confidence is less than 0.5, then... ,otherwise ;when When the device is in the recovery confirmation state, the drive control module outputs a recovery operation command, the transportation equipment starts running, and then returns to the normal operation state.

[0177] (3) When in fault lockout state, the drive control module outputs an emergency braking signal (braking deceleration of 0.3g to 0.5g) and blocks all start signals; the communication module 401 sends a first-level alarm information to the host computer control system, including obstacle type, size, location coordinates and timestamp; after manual obstacle clearing is completed, the host computer control system issues a remote reset command, and the main control module 201 releases the lockout and restores the system standby after receiving the reset command.

[0178] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for obstacle recognition and hierarchical control in hilly and mountainous terrain tracks based on improved YOLOv8, characterized in that, The process includes the following: Step 1: The transport equipment transports agricultural machinery on the track, and the industrial camera (102) collects image data in front of the track and transmits it to the main control module (201) in real time. Step 2: The main control module (201) inputs the preprocessed image into the improved YOLOv8 model to perform forward propagation inference; the improved YOLOv8 model includes four parts: StarNet backbone network, CAA attention module, FPN-PAN neck network, and detection head; the input image first passes through the StarNet backbone network for lightweight feature extraction, generating feature maps at three scales, and then inputs them into the CAA attention module for weight calibration to enhance the features of the track obstacle and suppress complex background interference. The calibrated feature maps are then fed into the FPN-PAN neck network for multi-scale feature fusion to obtain the fused feature maps at three scales, which are then input into the detection head. Finally, the category label, confidence score, and bounding box coordinates of the obstacle are output. Step 3: The main control module (201) performs hierarchical control of the transportation equipment based on the inference results output by the improved YOLOv8 model; Step 3.1: The track access area generation unit generates the track access area ROI based on the industrial camera calibration parameters; Step 3.2: The validity determination unit calculates the intersection-over-union ratio (IoU) of the obstacle bounding box and the ROI. ,like If it is an obstacle outside the track, it is considered an obstacle outside the track; otherwise, it is considered a valid obstacle on the track. Step 3.3: The size estimation unit further estimates the lateral width and height of the obstacle in the real world based on the bounding box pixel size and industrial camera calibration parameters, and compares them with the track clearance threshold and wheel radius threshold to determine whether it is a passable object or an impassable static obstacle. Step 3.4: The status control unit performs the switching and management of four states: normal operation, temporary avoidance, recovery confirmation, and fault lockout.

2. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 1, characterized in that, In step 2, the StarNet backbone network adopts a four-stage layered architecture, with each stage consisting of multiple stacked star operation modules. The data processing process of the star operation modules is as follows: Step 2.1.1: Input feature map Send in Depthwise separable convolutional layers are used for spatial feature extraction to obtain the output feature map of the depthwise separable convolution. ; Step 2.1.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Two are fed in parallel The convolutional layer expands the number of channels to four times the original number, obtaining output feature maps for two branches. , ; Step 2.1.3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation and Element-wise multiplication is performed to complete the star operation, resulting in the output feature map of the star operation. ; Step 2.1.4: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation The batch normalization layer performs normalization processing to obtain the batch normalized feature map. ; Step 2.1.5: Through Convolutional layers restore the number of channels to their original dimensions, obtaining feature maps with restored channels. ; Step 2.1.6: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Passing by again After depthwise separable convolution, it is compared with the original input feature map. The residues are added together to form a residual connection, which yields the output feature map of the star operation module. Step 2.1.7: Stack multiple star operation modules as described above, and then stack the first... The output of the star operation module is used as the first Input to the individual star operation module; Step 2.1.8: By stacking multiple star operation modules and using convolutional layers with a stride of 2 for downsampling, the StarNet backbone network finally outputs three scale feature maps: shallow feature map, medium feature map, and deep feature map.

3. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 1, characterized in that, In step 2, the CAA attention module introduces a dynamic strip convolution kernel size adjustment mechanism based on local texture complexity, enabling the receptive field of the strip convolution to adaptively adjust according to the content difference between the obstacle and the background. The specific data processing process is as follows: Step 2.2.1: For the three scale feature maps input to the StarNet backbone network, firstly through... The average pooling layer captures local region features, resulting in a feature map after average pooling. ; Step 2.2.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation pass The convolutional layer performs feature transformation to obtain the convolutional feature map. ; Step 2.2.3: To perceive the content differences at different spatial locations in the feature map, calculate the local texture complexity score for each location; first, ... pass Convolutional compression into single-channel feature maps ; then on Each spatial location Take a point centered on that location. Local window, Given a preset local window side length, calculate the variance of all pixel values ​​within that window as the local texture complexity score for that location. : In the formula, The coordinates of the center pixel for the current computational complexity score; These are the horizontal and vertical offsets relative to the center pixel within the local window, respectively, and their values ​​range from [ ]. , ]; For Centered The average value of all pixels within the window; Indicates rounding down; The complexity scores of all spatial locations constitute a complexity graph. ; Step 2.2.4: Based on the complexity graph For each spatial location, a set of attention weights is generated to fuse bar convolution kernels of multiple scales; First, set Candidate bar convolution kernels of different sizes, i.e., candidate convolution kernels, are given different receptive fields to capture short-range, medium-range, and long-range spatial contexts, respectively; the complexity graph is then used to... Each spatial location Complexity score Input a weight generator consisting of two fully connected layers, and output the weights corresponding to that position after Softmax normalization. Attention weights for each candidate convolutional kernel: In the formula, Spatial location place dimensional attention weight vector, Indicates spatial location Assigned to the The weights of each candidate convolutional kernel; and These are two fully connected layers; ReLU is the activation function; Softmax is the normalization function. Step 2.2.5: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation Enter them separately A number of parallel horizontal and vertical depth-separable strip convolutions, the first... The convolution uses the method defined in step 2.2.

4. One candidate convolutional kernel; Using the attention weights generated in step 2.2.4, for each spatial location of The outputs corresponding to each candidate convolutional kernel are weighted and fused to obtain the strip convolutional features of the adaptive receptive field; After weighted fusion, all spatial locations constitute a complete horizontal fused feature map. Feature maps fused with vertical direction ; Step 2.2.6: Fuse the strip convolution outputs from the two directions, and then... Convolutional layers and the sigmoid activation function generate attention weight maps ; Step 2.2.7: Calculate the attention weight map Element-wise multiplication with the three scale feature maps input from the StarNet backbone network yields refined feature maps at three scales along the channel dimension.

4. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 3, characterized in that, In step 2.2.5, the following will be performed: Enter them separately A number of parallel horizontal and vertical depth-separable strip convolutions, the first... The convolution uses the method defined in step 2.2.

4. One candidate convolutional kernel: In the formula, , They are respectively using the first The horizontal and vertical strip convolution outputs of the candidate convolution kernels; Indicates the kernel size as Depth-separable convolution, Indicates the kernel size as Depth-separable convolution; The strip convolutional features of the adaptive receptive field are as follows: In the formula, The fused horizontal strip convolution output feature map is located in space. eigenvalues ​​at that location The fused vertical strip convolution output feature map is located in space. Eigenvalues ​​at; To adopt the first The horizontal strip convolution of the candidate convolution kernels in spatial location Output feature value at, To adopt the first The vertical strip convolution of each candidate convolution kernel in spatial location The output feature value at that location.

5. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 1, characterized in that, In step 2, the improved YOLOv8 model is trained offline before deployment. The training dataset covers common obstacle types including falling rocks, fallen branches, branches crossing the track, and small animals. During training, data augmentation strategies including Mosaic enhancement, geometric transformation, color adjustment, and random flipping were employed to improve the model's generalization ability. The bounding box regression loss function used was a self-improved Wise-MPDIoU loss function, the calculation formula of which is as follows: In the formula, , These are the prediction boxes. The coordinates of the corner points; , The real frame number is 1. The coordinates of the corner points , To predict the width and height of the bounding box, As a penalty for aspect ratio consistency, It is a dynamic non-monotonic focusing factor. , For balance coefficient, It is a very small constant.

6. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 1, characterized in that, In step 3, if the validity determination unit determines that it is an obstacle outside the track, the transportation equipment continues to operate normally; if the validity determination unit determines that it is a valid track obstacle, the size estimation unit makes a further determination; when it is further determined to be a passable object, the status control unit controls the transportation equipment to switch to a temporary avoidance state; when it is further determined to be a non-passable static obstacle, the status control unit controls the transportation equipment to switch to a fault lock state.

7. The method for obstacle identification and hierarchical control in hilly and mountainous terrain according to claim 6, characterized in that, When the transport equipment is in a temporary obstacle avoidance state, the drive control module outputs a temporary braking signal, and the transport equipment decelerates and stops; the continuous frame confirmation unit sets a continuous obstacle-free frame counter. The initial value is 0. The industrial camera 102 continuously collects data. For each frame, if no obstacle is detected within the track travel area and the detection confidence is less than 0.5, then... ,otherwise ;when When the device is in the recovery confirmation state, the drive control module outputs a recovery operation command, the transportation equipment starts running, and then returns to normal operation.

8. The method for identifying and classifying obstacles on mountainous tracks according to claim 6, characterized in that, When the transport equipment is in a fault-locked state, the drive control module outputs an emergency braking signal and blocks all start signals; it sends a first-level alarm message to the host computer control system, including the type, size, location coordinates and timestamp of the obstacle; after the obstacle is cleared manually, the host computer control system issues a remote reset command, and the main control module (201) releases the lock and restores the system to standby mode after receiving the reset command.

9. A system for implementing the hilly and mountainous terrain track obstacle identification and hierarchical control method of claim 1, characterized in that, The system includes an image acquisition module (101), a main control module (201), a protective structure (301), and a communication module (401). The image acquisition module (101) includes an industrial camera (102) for acquiring image data in front of the track and transmitting it to the main control module (201) for analysis and processing, identifying obstacle categories, and controlling the transportation equipment to switch states. The protective structure (301) is used to provide physical protection for the core components of the system. The communication module (401) is used to transmit the recognition results of the main control module (201) to the outside. The main control module (201) integrates a drive control module and the following functional units: a track passage area generation unit, an effectiveness determination unit, a size estimation unit, a state control unit, a continuous frame confirmation unit, and an embedded AI calculation unit.

10. The system according to claim 9, characterized in that, The image acquisition module (101) is fixed to the transport equipment control box (701) by a mounting bracket (501), and the main control module (201) is fixed inside the transport equipment control box (701); the protective structure (301) is made of aluminum alloy and is installed on the transport equipment control box (701) and encloses the image acquisition module (101); the mounting bracket (501) includes a base (502), an adjusting arm (503), and a locking mechanism (504); the base (502) is fixed On the control box (701); the adjusting arm (503) is connected to the vertical rod of the base (502) through the universal joint (506) to realize the spatial angle adjustment of ±30°; the locking mechanism (504) acts on the rotating joint of the universal joint (506) through the wing nut (507), and applies a locking force to the rotating mating surface of the universal joint (506) by tightening the wing nut (507), so that the adjusting arm (503) is locked and fixed relative to the vertical rod; the industrial camera (102) is mounted on the adjusting arm (503).

Images