A method and system for remotely controlled vehicle road obstacle detection

Abstract

This invention relates to the field of vehicle detection technology and discloses a method and system for detecting road obstacles in remotely controlled vehicles. The method includes: acquiring road images and attaching frame rate metadata to generate image data packets; inputting them into an improved YOLOv8 backbone network to extract features, and obtaining denoised feature maps through dual-branch residual attention denoising; optimizing the output feature maps through downsampling and dual-coordinate attention noise reduction, combined with long-range dependency enhancement; adaptively adjusting the model structure and outputting detection results; and transmitting and generating control commands according to hazard levels. This invention solves the problems of insufficient detection accuracy and real-time performance caused by complex environmental noise interference, small target feature dilution, and frame rate and bandwidth fluctuations in remote control scenarios, achieving accurate and efficient obstacle detection for remotely controlled vehicles and ensuring driving safety.

Description

Technical Field

[0001] This invention relates to the field of vehicle detection technology, and in particular to a method and system for detecting road obstacles in remotely controlled vehicles. Background Technology

[0002] In remotely controlled vehicle applications, road obstacle detection typically relies on deep learning-based object detection models to identify various obstacles along the driving path, ensuring the safety of remote control. These technologies often employ a single-stage detection architecture, featuring high inference efficiency and convenient deployment. They are suitable for remote driving scenarios with high real-time requirements and are commonly found in enclosed or semi-enclosed operating environments such as mines and ports.

[0003] In actual operation, the communication and sensing environment of remotely controlled vehicles often suffers from severe image quality degradation. Specifically, in images captured by vehicle cameras under conditions of dust, strong electromagnetic interference, or drastic lighting changes, the edges and textures of small obstacles are severely obscured by noise. This image degradation is not a simple decrease in brightness or contrast, but rather a composite interference involving Gaussian random noise and pulsed salt-and-pepper noise, which suppresses high-frequency details in the original image. Since small target obstacles occupy a limited pixel area, their effective features are easily misjudged as background clutter by the model in noisy environments, thus failing to be effectively activated and responded to. The essence of this problem lies in the fact that existing detection models lack robust design for low signal-to-noise ratio inputs during the feature extraction stage, and their convolutional structures struggle to stably extract weak but crucial spatial gradient information under noise interference. The consequence of this is that remote operators may drive into areas with gravel or potholes without warning, which may cause minor tire damage or suspension system impact, or even vehicle rollover or brake failure. Especially on slopes or narrow work passages, such misjudgments may directly trigger a chain of safety accidents, threatening the safety of on-site equipment and personnel. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides a method for detecting road obstacles in remotely controlled vehicles, comprising: S1: Capture road images using a high-definition camera and generate image data packets by attaching frame rate metadata; S2: Input the image data packet into the improved YOLOv8 backbone network, extract the first feature map and the second feature map, and then process it through a dual-branch residual attention denoising process to output the denoised feature map after dynamic weighted fusion. S3: The denoised feature map is downsampled and optimized with dual-coordinate attention noise reduction, and combined with long-range dependency enhancement of shallow and deep features to output the optimized first feature map and second feature map; S4: Input the optimized first feature map and second feature map into the YOLOv8 feature fusion layer and the detection head, adaptively adjust the model structure according to the current frame rate level, and output obstacle detection results including position, size, category and confidence. S5: Combines obstacle detection results with transmission rate calculation, classifies them according to hazard level, and transmits them via UDP / TCP hybrid protocol to generate control commands that are fed back to the vehicle controller.

[0005] Furthermore, the steps for generating image data packets include: S11: Collects road image data through high-definition cameras; S12: Collect frame rate data via a performance counter; S13: Encapsulate the road image data and frame rate data into an image data packet; the structure of the encapsulated image data packet consists of a frame header, image data, frame rate metadata, and a checksum.

[0006] Further, the steps of extracting the first feature map and the second feature map and outputting a denoised feature map after denoising processing include: S21: Parse the image data packet to obtain the pixel tensor and frame rate metadata to be processed; the parsing includes confirming the data type through the data packet type identifier in the frame header, extracting the image data and frame rate metadata according to the data length field, and converting the image data into a three-dimensional pixel tensor. S22: Based on the improved YOLOv8 backbone network, extract the first feature map and the second feature map; the improved YOLOv8 backbone network is based on the original YOLOv8 architecture, and the core component is the CSPDarknet-53 network structure; S23: Input the first feature map and the second feature map into the dual-branch residual attention denoising module, perform dynamic weighted fusion, and output a denoised feature map; the dual-branch residual attention denoising module processes the feature map through two parallel branches, spatial variation noise suppression and contrast enhancement, and then performs dynamic weighted fusion based on noise intensity.

[0007] Furthermore, the steps for extracting the first and second feature maps based on the improved YOLOv8 backbone network include: S221: Adjust the number of channels of the tensor of the pixel to be processed through a 1×1 convolutional layer, and at the same time perform downsampling with a stride of 2 through a 3×3 convolutional layer to obtain the initial feature tensor; S222: The initial feature tensor is sequentially input into 5 sets of CSP residual modules, and processed progressively according to the rule of doubling the number of channels and halving the resolution, and the output is an intermediate feature tensor sequence containing 5 stage feature tensors; the CSP residual module superimposes the input feature map with the convolutional feature map through residual connection; S223: From the intermediate feature tensor sequence of the steps, extract the feature tensor after processing by the third group of CSP residual modules as the first feature map, and extract the feature tensor after processing by the fourth group of CSP residual modules as the second feature map.

[0008] Furthermore, the step of inputting the first feature map and the second feature map into the dual-branch residual attention denoising module to perform dynamic weighted fusion includes: S231: Perform spatial variation noise suppression branch processing to generate a denoised branch feature map; the spatial variation noise suppression branch adopts an improved HDRAB, which captures noise distributions at different scales through 3-level dilated convolution, and introduces residual connections at the same time; S232: Perform contrast enhancement branch processing to generate an enhanced branch feature map; the contrast enhancement branch is based on Retinex theory, decomposes the feature map into illumination component and reflection component, and performs illumination compensation on the reflection component of low illumination area. S233: Calculate noise intensity to provide a basis for weighted fusion; S234: Generate an attention fusion weight matrix based on noise intensity, dynamically weight and fuse the denoised branch feature map and the enhanced branch feature map, and output the denoised feature map.

[0009] Furthermore, the steps of downsampling and dual-coordinate attention noise reduction optimization of the denoised feature map include: S31: Input the denoised feature map into the spatial pyramid pooling-fast Fourier convolution module, perform downsampling operation, and generate deep feature maps and ultra-deep feature maps; the spatial pyramid pooling-fast Fourier convolution module combines spatial pyramid pooling and fast Fourier convolution, first performs preliminary downsampling, then performs spatial pyramid pooling through multi-scale pooling kernels, and finally decomposes low-frequency components and high-frequency components and concatenates them; S32: Input the deep feature map and the ultra-deep feature map into the dual-coordinate attention noise-resistant feature extraction module, perform noise-resistant optimization and long-range dependency enhancement, and output the optimized first feature map and the optimized second feature map.

[0010] Furthermore, the steps for inputting the deep feature map and the ultra-deep feature map into the dual-coordinate attention-based noise-reducing feature extraction module for optimization include: S321: Execute a parallel coordinate attention mechanism to generate a coordinate attention feature map; the parallel coordinate attention mechanism constructs attention weight maps in the x-axis and y-axis directions respectively, and multiplies the deep feature map with the attention weight map pixel by pixel; S322: Perform noise-aware downsampling optimization to generate a noise-aware downsampling feature map; the noise-aware downsampling optimization first calculates the local noise intensity through the noise detection window, selects median filtering preprocessing or direct average pooling according to the noise intensity, and adjusts the channel segmentation ratio of the feature tensor at the same time. S323: Perform long-range dependency enhancement to generate an enhanced ultra-deep feature map; the long-range dependency enhancement adjusts the first feature map to the same resolution as the ultra-deep feature map through multi-level attention propagation, and then transmits shallow detail information to deep features through a cross-attention mechanism; S324: Define the noise-aware downsampled feature map as the optimized first feature map and the enhanced ultra-deep feature map as the optimized second feature map.

[0011] Furthermore, the steps for adaptively adjusting the model structure and output obstacle detection results include: S41: Determine the frame rate level based on frame rate metadata and adaptively adjust the YOLOv8 model structure; the adaptive adjustment is performed by the frame rate-aware multi-mode switching module, including calculating the average frame rate, dividing into high, medium and low frame rate levels, and adjusting the input resolution, the number of attention module activations and the number of candidate boxes accordingly. When switching frame rate modes, the model parameters are smoothed by the exponential moving average algorithm. S42: Perform feature fusion and detection inference, and output obstacle detection results; the feature fusion is performed through the YOLOv8 feature fusion layer to perform bidirectional feature fusion, and the detection inference calculates the obstacle category probability through the classification branch and calculates the obstacle bounding box coordinates and confidence through the regression branch.

[0012] Furthermore, the steps of transmitting obstacle detection results in a hierarchical manner and generating control commands include: S51: Calculate the average transmission bit rate and determine the bit rate level; The calculation of the average transmission bit rate is performed by the bit rate adaptive feature split transmission module through a sliding window algorithm, and high, medium and low bit rate levels are divided according to the average transmission bit rate. S52: Based on the bit rate level, formulate a hierarchical transmission strategy; the hierarchical transmission strategy selects to transmit complete feature data, selectively transmit feature data, or transmit only core feature data according to different bit rate levels; S53: Assess the hazard level of small targets and transmit data via a hybrid UDP / TCP protocol; the hazard level of small targets is calculated based on a weighted average of size factor, distance factor, and velocity factor, with the highest priority data transmitted using the TCP protocol and the medium and low priority data transmitted using the UDP protocol. S54: Generate control commands and feed them back to the vehicle controller; the generated control commands include remote cockpit visualization of obstacle information, automatic generation of preliminary control commands, which are then adjusted and fed back to the vehicle controller to be converted into actuator control signals.

[0013] Compared to existing technologies, the advantages of this invention are as follows: This invention addresses the core problems faced by remotely controlled vehicles in complex scenarios such as mining areas and ports, including complex noise interference, dilution of small target features, and imbalances in detection accuracy and real-time performance due to frame rate and bandwidth fluctuations. It achieves accurate and efficient detection through targeted technical design. The improved YOLOv8 backbone network's CSPDarknet-53 structure, through cross-stage local connections and residual connections, avoids the loss of small target features caused by repeated convolutions in traditional networks. This ensures that the edge texture features of small obstacles such as gravel and bolts are fully preserved, solving the problem of difficulty in identifying small targets due to feature dilution.

[0014] The dual-branch residual attention denoising module employs parallel branches for spatial variation noise suppression and contrast enhancement. This precisely filters out composite noise such as Gaussian noise and impulse noise, while simultaneously enhancing the grayscale difference between small targets and the background in low-light areas. This addresses the issues of noise masking details and feature blurring caused by uneven illumination, significantly improving the clarity of feature extraction in complex environments. Long-range dependency enhancement technology constructs a mapping between shallow detail features and deep semantic features, supplementing missing small target details in ultra-deep features. This avoids confusion between small targets and the background during deep semanticization, ensuring the accuracy of target recognition.

[0015] The frame rate adaptive adjustment mechanism dynamically switches model modes based on the computing power status of edge computing nodes. When computing power is limited, a lightweight design ensures real-time detection, while improving detection accuracy when computing power is sufficient, achieving a dynamic balance between accuracy and speed. A bandwidth-tiered transmission strategy combined with a UDP / TCP hybrid protocol, along with small target hazard level assessment, prioritizes the transmission of critical data from high-risk targets during bandwidth fluctuations, saving bandwidth resources while ensuring no loss of critical detection information. In summary, this invention effectively addresses the core pain points of obstacle detection in remote control scenarios, providing reliable support for the driving safety of remotely controlled vehicles. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a remote-controlled vehicle road obstacle detection method according to the present invention; Figure 2 This is a schematic diagram of frame rate level-driven adaptive adjustment of YOLOv8 model structure and image resolution optimization in an embodiment of the present invention; Figure 3This is a flowchart illustrating the small target hazard level assessment and UDP / TCP hybrid protocol hierarchical transmission process in an embodiment of the present invention; Figure 4 This is a functional block diagram of a remote-controlled vehicle road obstacle detection system according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Please see Figure 1 As shown, this embodiment provides a method for detecting road obstacles by a remotely controlled vehicle, including: Step S1: Capture road images using a high-definition camera and attach frame rate metadata to generate an image data packet.

[0020] This step provides foundational data for road obstacle detection in remotely controlled vehicles. By synchronously collecting visual and computing power status data and standardizing and packaging it, the timing misalignment problem caused by the separation of image and frame rate data is resolved. The high-definition camera is an automotive-grade image acquisition device with dustproof, waterproof, and electromagnetic interference resistance capabilities. The lens is equipped with an anti-glare coating and a dust cover, enabling it to operate stably in complex environments such as mining areas and ports, acquiring visual information about road surfaces and obstacles.

[0021] Specifically, the steps for acquiring road images and attaching frame rate metadata to generate image data packets are as follows: Step S11: Collect road image data using a high-definition camera.

[0022] High-definition cameras capture road images at a preset frame rate, and adjust parameters through adaptive exposure and automatic white balance algorithms to avoid overexposure, underexposure, and color cast, thus obtaining road image data.

[0023] Step S12: Collect frame rate data using performance counters.

[0024] The performance counter collects inference time in real time, and after temporary storage, calculates the average inference time to form frame rate data.

[0025] Step S13: Encapsulate the road image data and frame rate data into an image data packet.

[0026] The two types of data are encapsulated into an image data packet, with the following structure in sequence: frame header, image data, frame rate metadata, and checksum: The frame header is 16 bytes, containing data type identifier, total length, and GNSS synchronization timestamp; the image data adopts YUV420 format, with resolution adapted to the detection field of view requirements to ensure coverage of small target pixel areas; the frame rate metadata is 8 bytes, containing single frame and average inference time; the checksum is 4 bytes, calculated using the CRC32 algorithm, and used to verify data integrity.

[0027] The generated image data package can be directly used as the input variable in step S2, providing time-consistent basic data support for subsequent feature extraction and obstacle detection.

[0028] Step S2: Input the image data packet into the improved YOLOv8 backbone network, extract the first feature map and the second feature map, and perform dual-branch residual attention denoising processing to output the dynamically weighted fused denoised feature map.

[0029] This step addresses the feature extraction and denoising requirements under low signal-to-noise ratio conditions in remote control scenarios. It calls the image data packet generated in step S1, extracts features through an improved YOLOv8 backbone network, and then performs dual-branch residual attention denoising to solve the problems of feature dilution of small targets and interference from compound noise, providing high-quality input for subsequent feature optimization.

[0030] Specifically, the steps for extracting features and outputting a denoised feature map after denoising are as follows: Step S21: Parse the image data packet to obtain the pixel tensor and frame rate metadata to be processed.

[0031] The image data packets generated in step S1 are input into the improved YOLOv8 backbone network. First, a data packet parsing operation is performed: the receiving end confirms that the data type is road obstacle detection data through the data packet type identifier in the frame header, and then extracts the subsequent image data and frame rate metadata according to the data length field. The frame rate metadata is temporarily stored in the cache unit of the edge computing node (for frame rate adaptation processing in step S4). The image data is converted into a three-dimensional pixel tensor (3 channels, height H, width W) through the YUV420 to RGB algorithm. This conversion adjusts the pixel value mapping relationship of the RGB channels through a pre-trained color mapping matrix to ensure that the converted pixel tensor can accurately reflect the color contrast of the original image (such as the grayscale difference between gravel and road surface), avoiding color distortion from affecting subsequent feature extraction. Finally, the pixel tensor to be processed is obtained as the input data of the improved YOLOv8 backbone network.

[0032] Step S22: Extract the first feature map and the second feature map based on the improved YOLOv8 backbone network.

[0033] The improved YOLOv8 backbone network is a feature extraction network optimized for the low signal-to-noise ratio characteristics of remote control scenarios, based on the original YOLOv8 architecture. The core component is the CSPDarknet-53 network structure. This structure divides the feature map into two branches through cross-stage local connections (CSP). One branch performs convolution operations to extract semantic features through multiple sets of residual blocks (each set contains 3 3×3 convolutional layers and 1 residual connection). The other branch directly performs downsampling with a stride of 2 and then concatenates it with the former. This reduces the amount of computation while retaining more detailed features of small targets (such as the edge texture of gravel), avoiding the problem of dilution of small target features caused by repeated convolutions in the traditional Darknet network.

[0034] Specifically, the feature extraction process is as follows: Step S221: Adjust the number of channels of the tensor of the pixel to be processed from 3 to 32 through a 1×1 convolutional layer, and at the same time perform downsampling with a stride of 2 through a 3×3 convolutional layer to obtain an initial feature tensor with 32 channels and a resolution of H / 2×W / 2.

[0035] Step S222: The number of channels of the feature map is gradually increased and the resolution is reduced by using 5 groups of CSP residual modules (groups 1 to 5 contain 1, 2, 8, 8 and 4 residual blocks respectively). Each group of CSP residual modules superimposes the input feature map and the convolutional feature map through residual connections to alleviate the gradient vanishing problem (especially gradient decay during deep feature extraction).

[0036] Step S223: After processing by the third group of CSP residual modules, the first feature map is output, and after processing by the fourth group of CSP residual modules, the second feature map is output. The first feature map is a shallow feature map (256 channels, resolution H / 8×W / 8), corresponding to the low-level semantic information of the image. It contains rich details such as edges and textures (e.g., the outline edges of gravel, grayscale variations of potholes), reflecting the basic shape of small target obstacles. The second feature map is a mid-level feature map (512 channels, resolution H / 16×W / 16), corresponding to the mid-level semantic information of the image. Through the aggregation and abstraction of shallow features, it initially distinguishes obstacles from the background area (e.g., separating the grayscale features of gravel from the road surface texture). However, due to noise interference (e.g., grayscale fluctuations caused by Gaussian noise), feature blurring still exists, requiring further denoising optimization.

[0037] Step S23: Input the first feature map and the second feature map into the dual-branch residual attention denoising module (DB-RA-DNM module), perform dynamic weighted fusion, and output the denoised feature map.

[0038] The DB-RA-DNM module is a feature optimization module designed for composite noise (Gaussian noise + impulse noise) in remote control scenarios. It addresses the problems of "noise masking detailed features" and "uneven lighting leading to low feature contrast" through two parallel branches: spatial variation noise suppression and contrast enhancement. Then, based on dynamic weighted fusion of noise intensity, it outputs a denoised feature map. Specifically, the denoising and fusion process is as follows: Step S231: Perform spatial variation noise suppression branch processing to generate a denoised branch feature map.

[0039] This branch employs an improved HDRAB (Hybrid Dilated Residual Attention Block), the core of which is to capture noise distributions at different scales through three levels of dilated convolutions (dilation rates of 1, 3, and 5). A 3×3 convolution with dilation rate 1 is used to filter pixel-level Gaussian noise (such as random grayscale fluctuations caused by electromagnetic interference in mining areas), and its kernel parameters are determined through training with a Gaussian noise sample library to ensure suppression of Gaussian noise in the 0–50 dB range. A 3×3 convolution with dilation rate 3 is used to filter small-area impulse noise (such as isolated bright spots caused by dust adhering to the lens), expanding the receptive field to cover a noise area of ​​2–3 pixels. A 3×3 convolution with dilation rate 5 is used to filter large-area noise (such as stripe noise caused by sudden changes in illumination at tunnel entrances and exits), covering a noise area of ​​5–7 pixels. Simultaneously, residual connections are introduced, directly superimposing the input first / second feature map with the feature map processed by the three levels of dilated convolutions, using the following formula: ; in, For the denoising branch feature map, This represents either the first feature map or the second feature map. The algorithm employs a three-stage dilated convolutional combination (3×3 convolutions with dilation rates of 1, 3, and 5 are sequentially concatenated, followed by a batch normalization (BN) layer and a SiLU activation function after each convolutional layer). The BN layer accelerates training convergence by standardizing feature values ​​(adjusting the feature mean to 0 and the variance to 1). The SiLU activation function is a smooth, non-linear activation function whose core characteristic is multiplying the input value with the output of the sigmoid function, effectively mitigating the gradient vanishing problem, enhancing feature representation, and avoiding insufficient feature discrimination caused by linear activation. This operation yields a denoised branch feature map, which reduces the intensity of composite noise while preserving the edge details of small objects.

[0040] Step S232: Perform contrast enhancement branch processing to generate enhanced branch feature maps.

[0041] This branch is based on Retinex theory. It first decomposes the input first / second feature map into illumination and reflection components, as shown in the formula: ; in, This represents the pixel value at position (x, y) in either the first or second feature map. This refers to the illumination component (reflecting the overall brightness distribution, such as low-light areas inside a tunnel or high-light areas in the open air). The reflection component (reflects the essential characteristics of the target, such as the grayscale texture of gravel). The standard deviation of the Gaussian kernel is calculated by the Gaussian filtering algorithm and dynamically adjusted by the local brightness variance of the image. The standard deviation is increased (e.g., 15) in areas where the brightness variance is greater than a preset threshold (e.g., 0.1) (e.g., road surface reflection under strong light) to smooth the lighting, while the standard deviation is decreased (e.g., 5) in areas where the brightness variance is less than a preset threshold (e.g., shadows in a tunnel) to preserve details. pass and It is obtained by pixel-by-pixel division, that is .

[0042] against Low-light areas below the illumination threshold (determined by statistically analyzing the lowest illumination intensity at which small targets can be clearly identified in a remotely controlled scene, such as 0.3) are subject to... Illumination compensation is performed, with the compensation coefficient being the dynamic enhancement coefficient. The formula is: ; in, This represents the reflected component after illumination compensation; The calculation formula is: ; in, k The enhancement coefficient adjustment coefficient is determined through training with a remote-controlled scene noise sample library (range 1.2~1.8), controlling the rate of change of the enhancement coefficient. The calculation formula is an improved formula, based on the following: The Sigmoid function is widely used in adaptive gain control, with traditional applications including automatic exposure and adaptive contrast. A typical form is... , is a standard nonlinear mapping in cybernetics and neural networks. Where: t is the input variable in traditional adaptive gain control scenarios, representing physical quantities that need adaptive adjustment, such as brightness, local contrast, and signal strength; t0 is the threshold parameter, the critical value of the input variable t, used to divide the input variable's interval and determine the inflection point of the output z; β is the rate of change adjustment coefficient, taking the value of a positive real number (typically in the range of 1~5). The larger β is, the faster the output z changes with the input t, and the steeper the curve; z is the output value, taking the value range of [0,1], usually used to represent the adaptive gain coefficient, weight value, or probability value, realizing the nonlinear mapping from the input variable to the target interval. The improvement lies in: defining the input variable as noise intensity N (instead of the traditional brightness or contrast), and the output as the enhancement coefficient of the Retinex reflection component; by setting Noise threshold ( The value was determined through detection accuracy verification using a noise sample library for remote control scenarios, ensuring that the noise intensity is higher than [the specified value]. (When the enhancement factor decreases), a counterintuitive but reasonable strategy is implemented: "the greater the noise, the weaker the enhancement." This is applicable when the noise intensity N is higher than... hour, The value should approach 0.5 to avoid excessive amplification of the reflection component, which would lead to noise amplification; when the noise intensity N is lower than 0.5... hour, Approaching 1, the reflection component is significantly enhanced to improve the contrast in low-light areas (such as the grayscale difference between gravel and road surface inside a tunnel). This operation yields an enhanced branch feature map, in which the contrast of small target features in low-light areas is significantly improved.

[0043] Step S233: Calculate the noise intensity N to provide a basis for weighted fusion.

[0044] The formula for calculating noise intensity N is:

[0045] Where H is the height of the input feature map (in pixels); W is the width of the input feature map (in pixels). This formula is an improved formula, based on the following: commonly used image noise assessment metrics include local variance method, gradient magnitude method, and signal-to-noise ratio estimation. The improvement of this invention lies in: using gradient magnitude... (Molecular) and Local Variance Construct the "margin saliency / noise flatness" ratio (denominator), where Calculate using the Sobel operator This reflects the edge strength of the feature map; where ∇B(x,y) represents the gradient magnitude of the input feature map at pixel coordinates (x,y); ∇ is the gradient operator (Nabla operator), used to represent the rate of change of the function in the spatial dimension, specifically referring to the gradient of the grayscale change of the feature map pixel value in the two-dimensional plane (xy coordinates); ∇ x B(x,y) represents the gradient component of B(x,y) in the x-direction, reflecting the rate of grayscale change of the pixel in the horizontal direction; ∇ y B(x,y) is the gradient component of B(x,y) in the y direction, reflecting the rate of gray-level change of the pixel in the vertical direction; The local variance within the 3×3 sliding window reflects the local grayscale fluctuations of the feature map. This indicates a window convolution operation (smoothing local variance fluctuations through 3×3 mean convolution), enhancing robustness. This ratio effectively distinguishes true edges (high gradient + high variance, ratio close to 1) from impulse noise (high gradient + low variance, ratio much greater than 1), solving the problem of "misclassifying impulse noise as edges" in traditional noise assessment, making noise intensity calculation more accurate. This formula yields the noise intensity N corresponding to the first and second feature maps.

[0046] Step S234: Generate an attention fusion weight matrix based on the noise intensity N, and dynamically weight and fuse the denoised branch feature map and the enhanced branch feature map.

[0047] The size of the attention fusion weight matrix is ​​related to the first feature map / second feature map. Figure 1 The weight value for each location is calculated using the noise intensity N: For regions where the noise intensity N is higher than the fusion threshold (determined by the "noise intensity - detection accuracy" curve, i.e., the noise intensity value at which detection accuracy begins to decrease significantly), a higher denoising branch weight (e.g., 0.7~0.9) and a lower enhancement branch weight (e.g., 0.1~0.3) are assigned to prioritize preserving the clear features filtered by the denoising branch (e.g., the contours of gravel after suppressing dust noise); for regions where the noise intensity N is lower than the fusion threshold, a higher enhancement branch weight (e.g., 0.6~0.8) and a lower denoising branch weight (e.g., 0.2~0.4) are assigned to strengthen the contrast features enhanced by the enhancement branch (e.g., the difference in gravel density in low-light areas). The weighted fusion formula is: ; in, This is the output denoised feature map; For attention fusion weight matrix, For the denoising branch feature map, To enhance the branch feature map, "×" indicates pixel-wise multiplication, and "+" indicates pixel-wise addition. Through this fusion operation, a denoised feature map is output, which suppresses compound noise and improves the contrast of small target features, providing high-quality input for feature optimization in step S3.

[0048] Step S3: The denoised feature map is downsampled and optimized with dual-coordinate attention noise reduction, and combined with long-range dependency enhancement of shallow and deep features to output the optimized first feature map and second feature map.

[0049] This step focuses on the needs of small target feature enhancement and deep feature optimization. It calls the denoised feature map generated in step S2 and solves the problems of "small target features being submerged by deep semantic features" and "noise causing deep feature attenuation" through downsampling and dual-coordinate attention noise reduction optimization, providing optimized feature input for the detection inference in step S4.

[0050] Specifically, the steps for downsampling, noise reduction optimization, and long-range dependency enhancement are as follows: Step S31: Input the denoised feature map into the SPP-FFC module, perform downsampling operation, and generate deep feature map and ultra-deep feature map.

[0051] The downsampling module is an SPP-FFC module, which combines the advantages of Spatial Pyramid Pooling (SPP) and Fast Fourier Convolution (FFC). While reducing feature map resolution and computational load, it retains key semantic features and global details, adapting to the computational limitations of edge computing nodes (such as embedded GPUs) in remote control scenarios (typically with less than 10 TOPS). Specifically, the downsampling process is as follows: Step S311: Perform preliminary downsampling on the denoised feature map using a 3×3 convolution with a stride of 2 to reduce the spatial dimension of the feature map (the resolution becomes 1 / 2 of the original), while increasing the number of channels to twice the original (e.g., when the number of channels in the denoised feature map is 256, the number of channels after preliminary downsampling is 512), thereby enhancing the semantic expressive power of the feature map (e.g., the semantic features that distinguish between gravel and potholes).

[0052] Step S312: Perform spatial pyramid pooling operation on the feature map after preliminary downsampling using multi-scale pooling kernels (pooling kernel sizes are 1×1, 3×3, and 5×5 respectively), and concatenate the pooling results of different scales (corresponding to features of 1×1, 3×3, and 5×5 receptive fields respectively) into a multi-scale feature tensor.

[0053] Step S313: The multi-scale feature tensor is decomposed into low-frequency components and high-frequency components through the FFC module. The low-frequency components are used to extract global semantic features (such as the overall outline of obstacles) through traditional 3×3 convolution. The high-frequency components are transformed to the frequency domain through two-dimensional fast Fourier transform (FFT) to extract local detail features (such as the texture frequency of gravel).

[0054] Step S314: The low-frequency component and the high-frequency component are transformed back to the spatial domain by inverse FFT and then concatenated to obtain a deep feature map (1024 channels, resolution H / 32×W / 32) and an ultra-deep feature map (2048 channels, resolution H / 64×W / 64), which provide multi-scale feature input for subsequent dual-coordinate attention noise reduction optimization.

[0055] Step S32: Input the deep feature map and the ultra-deep feature map into the dual-coordinate attention noise-resistant feature extraction module (DCAFE-AN module), perform noise-resistant optimization and long-range dependency enhancement, and output the optimized first feature map and the optimized second feature map.

[0056] The DCAFE-AN module enhances the features of small target regions and supplements detailed information through three sub-steps: parallel coordinate attention mechanism, noise-aware downsampling optimization, and long-range dependency enhancement. Specifically, the optimization process is as follows: Step S321: Execute the parallel coordinate attention mechanism to generate a coordinate attention feature map.

[0057] This mechanism constructs attention weight maps along the x and y axes respectively. By focusing on the features of the coordinate region where the small target is located, it enhances the response intensity of the small target's features (e.g., amplifying feature values ​​in a gravel region), avoiding the dilution of small target features caused by global pooling in traditional spatial attention. Specifically, the x-axis attention weights... The calculation formula is: ; Y-axis attention weight The calculation formula is: ; In the formula, For deep feature maps, h is the channel index (value range 1~1024), h is the height index (value range 1~H / 32), and w is the width index (value range 1~W / 32). It is a fully connected layer (1024 input channels, 1024 output channels, with ReLU activation function to enhance non-linearity). The activation function maps weights to a range of 0-1, with larger weights indicating greater importance of the feature in that region. During calculation, the deep feature map is first averaged along the height direction (H) to obtain the global features along the X-axis (reflecting the aggregated feature information of different rows within the same column, such as the vertical distribution of gravel under the same x-coordinate). This is then used with a fully connected layer and a sigmoid function to generate the X-axis attention weight map. Similarly, the global features along the width direction (W) are averaged to obtain the global features along the Y-axis (reflecting the aggregated feature information of different columns within the same row, such as the horizontal distribution of gravel under the same y-coordinate), generating the Y-axis attention weight map. Finally, the deep feature map is multiplied pixel-by-pixel with the X-axis and Y-axis attention weight maps to obtain the coordinate attention feature map. The formula is: ; Here, · represents pixel-wise multiplication. Through this operation, the feature response intensity of small target regions in the coordinate attention feature map is increased by 2 to 3 times, while the features of background regions are suppressed, providing focused feature input for subsequent noise-aware downsampling.

[0058] Step S322: Perform noise-aware downsampling optimization to generate a noise-aware downsampling feature map.

[0059] First, a 3×3 noise detection window is enabled before the average pooling operation, and the coordinate attention feature map is then processed. For each pixel in the calculation, the local noise intensity within its window is calculated using the noise intensity calculation formula in step S233. When the local noise intensity is higher than the downsampling noise threshold (the downsampling noise threshold is determined by the "noise intensity - downsampling distortion rate" curve, i.e., the noise intensity value when the feature distortion rate begins to rise significantly after downsampling), the pixels within that window are preprocessed using median filtering, with the formula as follows: ; in, It is the pixel value at (x,y) of the feature map after median filtering; The median filtering function (taking the median of 9 pixel values ​​within the window) suppresses isolated high-brightness noise points (such as pixel value abrupt changes caused by electromagnetic interference), preventing noise from spreading during pooling. When the local noise intensity is lower than the downsampling noise threshold, average pooling is directly performed (pooling kernel size 2×2, stride 2) to preserve the smoothness of features (such as the continuity of gravel contours). Simultaneously, the channel segmentation ratio of the feature tensor is adjusted from the traditional 1:1 to 3:2, meaning 3 / 5 of the channels are used to preserve edge texture features (key channels for small object detection, such as channels reflecting the edges of gravel), and 2 / 5 of the channels are used to preserve semantic features (such as channels distinguishing obstacle types). This adjustment is determined through channel importance evaluation (based on the feature contribution of small object detection, using a random forest algorithm to calculate the impact of each channel on the accuracy of small object recognition), ensuring that sufficient small object edge details are retained after downsampling, avoiding the problem of "over-compression of edge channels" caused by traditional equal-ratio segmentation. Through this operation, a noise-aware downsampling feature map is obtained. (Number of channels: 1024; Resolution: H / 64×W / 64).

[0060] Step S323: Perform long-range dependency enhancement to generate enhanced ultra-deep feature maps.

[0061] This step, through multi-level attention propagation, associates the first feature map (shallow feature map, resolution H / 8×W / 8) output in step S223 with the ultra-deep feature map (resolution H / 64×W / 64), establishing a mapping relationship between shallow detail features and deep semantic features, and supplementing the small target detail information lost due to noise in the deep features (such as blurred gravel texture in the ultra-deep features). Specifically, the long-range dependency enhancement process is as follows: Step S3231: Adjust the first feature map to the same resolution as the ultra-deep feature map by downsampling with a step size of 8 (using bilinear interpolation downsampling to avoid feature distortion), thus obtaining a shallow feature map with adapted resolution. .

[0062] Step S3232: Calculate the similarity matrix between the shallow feature map and the ultra-deep feature map at the adapted resolution using a cross-attention mechanism, thus transferring shallow detail information to the deep features. The formula is as follows: ; in This refers to a very deep feature map. Cross-Attention is a cross-attention mechanism; this formula is an improved formula, the improvement being that it explicitly uses the shallow features (the first feature map) as the query. ), ultra-deep features as Key / Value ( This achieves information backfeeding from "shallow details to deep semantics" through cross-attention, rather than the traditional top-down fusion from "deep semantics to shallow"; weights are introduced. ( The value of is determined through a validation set of small target detection in remote control scenarios, and a grid search method is used to find the value that maximizes the mAP of small target detection. The value controls the fusion strength to avoid interference from shallow noise in deep semantic features (e.g., dust noise in the shallow layer will not excessively affect deep semantics), ensuring that the fusion both supplements the details of small targets and does not destroy the accuracy of deep semantics (e.g., it can still accurately distinguish between gravel and potholes). Through this operation, an enhanced ultra-deep feature map is obtained. (Number of channels: 2048; resolution: H / 64×W / 64).

[0063] Step S324: Define the optimized first feature map and the optimized second feature map.

[0064] noise-sensing downsampled feature map Defined as the first feature map for optimization, it will enhance the ultra-deep feature map. The optimized second feature map is defined as follows: the optimized first feature map retains the edge texture features of small targets (such as the edges of gravel), and the optimized second feature map has enhanced detail information and semantic discrimination ability (such as being able to identify the texture of gravel and determine its category). Together, they serve as the feature fusion and detection inference input for step S4, ensuring the accurate detection of small target obstacles.

[0065] For example, when a remotely controlled port vehicle travels to the container stacking area, the optimized first feature map clearly preserves the edge contours of the scattered bolts (small targets) on the ground, and the optimized second feature map supplements the texture details of the bolts through long-range dependency enhancement, avoiding confusion between the bolts and ground debris caused by ultra-deep feature semanticization, and providing accurate feature support for subsequent detection and inference.

[0066] Step S4: Input the optimized first and second feature maps into the YOLOv8 feature fusion layer and the detection head. Adaptively adjust the model structure according to the current frame rate level, and output obstacle detection results including location, size, category, and confidence. Please refer to [link to relevant documentation]. Figure 2 .

[0067] This step focuses on the needs of frame rate fluctuation adaptation and accurate detection in remote control scenarios. It calls the optimized first feature map and optimized second feature map generated in step S3, as well as the frame rate metadata temporarily stored in step S21. Through adaptive model adjustment and detection inference, it solves the problem that the traditional fixed parameter detection mode cannot adapt to frame rate fluctuations and outputs accurate obstacle detection results.

[0068] Specifically, the steps for adaptively adjusting the model structure and detection inference are as follows: Step S41: Determine the frame rate level based on frame rate metadata and adaptively adjust the YOLOv8 model structure.

[0069] The frame rate metadata temporarily stored in the edge computing node cache unit in step S21 is invoked. The frame rate-aware multi-mode switching module (FR-MSM module) calculates the current average frame rate based on the frame rate metadata and adaptively adjusts the working mode of the YOLOv8 model. The core function of the FR-MSM module is to dynamically adjust the model's input resolution, the number of activated attention modules, and the number of candidate boxes by monitoring the frame rate status in real time. This achieves a dynamic balance between detection accuracy and inference speed, ensuring real-time detection (frame rate ≥ 15fps) and continuity (position offset < 3 pixels) in remote control scenarios. Specifically, the process of frame rate level determination and model adjustment is as follows: Step S411: Calculate the average frame rate .

[0070] The FR-MSM module first calculates the average frame rate based on frame rate metadata. The formula is: ; in, This formula represents the inference time (in seconds) of 10 consecutive frames recorded in the frame rate metadata. It is a standard frame rate calculation method that averages the inference time of 10 frames to reduce frame rate judgment errors caused by fluctuations in single-frame inference time (such as extended single-frame times due to sudden increases in computing power), thus ensuring an average frame rate. It can accurately reflect the short-term computing power status of edge computing nodes.

[0071] Step S412: Determine the frame rate level.

[0072] Based on average frame rate The current frame rate level is determined by the threshold for classifying frame rate levels, which is based on the upper limit of the computing power of edge computing nodes (such as the maximum inference frame rate of embedded GPUs) and the real-time requirements of remote control scenarios (such as remote control requiring a frame rate ≥ 15fps to avoid decision latency). The specific determination rules are as follows: When When the frame rate exceeds the high frame rate threshold (e.g., 25fps), it is determined to be in high frame rate mode (the node has sufficient computing power to prioritize detection accuracy); when When the frame rate falls between the high frame rate threshold and the low frame rate threshold (e.g., 15fps), it is determined to be in medium frame rate mode (node ​​computing power is moderate, requiring a balance between accuracy and speed); when When the frame rate is below the low frame rate threshold, it is determined to be in low frame rate mode (node ​​computing power is tight, and real-time performance needs to be prioritized).

[0073] Step S413: Adjust the model structure according to the frame rate level.

[0074] For different frame rate modes, the FR-MSM module adaptively adjusts the YOLOv8 model structure, ensuring that the optimized first feature map and optimized second feature map can adapt to the adjusted model input requirements during the adjustment process. 1. High Frame Rate Mode Adjustment: Enable the full YOLOv8 model structure, maintaining the input resolution consistent with the optimized first feature map resolution (H / 64×W / 64), without requiring downsampling compression; activate the ACmix hybrid attention module (combining channel and spatial attention, adjusting channel weights through 1×1 convolutions and spatial weights through 3×3 convolutions to enhance the response of small target features) in the P3 / P4 / P5 branches, with 100% activation (all attention heads are enabled); set the number of small target detection candidate boxes to a preset high value (e.g., 300), increasing the candidate box coverage density (e.g., generating 5 candidate boxes per 10×10 pixel area) to ensure sufficient coverage of small targets (e.g., 5×5~15×15 pixel gravel), avoiding missed detections due to insufficient candidate boxes. This mode is suitable for scenarios with low edge computing node load (e.g., only a single mining card online in a mining area) and sufficient inference speed, fully leveraging the model's detection performance, achieving a small target detection mAP@0.5≥0.92, meeting the accurate detection requirements in low signal-to-noise ratio environments.

[0075] 2. Medium Frame Rate Mode Adjustment: Reduce the input resolution to 80% of the high frame rate mode (e.g., if the optimized first feature map resolution is H / 64×W / 64, adjust it to H / 80×W / 80). Reduce the number of pixels in the input feature map through bilinear interpolation downsampling, reducing inference computation (reducing computation by approximately 36%). Turn off 40% of the ACmix attention heads in the P3 / P4 / P5 branches, retaining attention heads that contribute highly to small object detection (retaining the top 60% of attention heads based on feature importance evaluation), reducing the resource consumption of attention computation (reducing attention computation time by approximately 40%). Adjust the number of small object candidate boxes to a preset median (e.g., 200). By reducing the candidate box coverage density (e.g., generating 3 candidate boxes per 10×10 pixel area), while ensuring that most small objects are covered, reduce the computation time for candidate box selection and bounding box regression (reducing detection head computation time by approximately 33%). This mode is suitable for scenarios with moderate load on edge computing nodes (such as 3 to 5 mining cards online at the same time in a mining area). It achieves a small target detection mAP@0.5≥0.88 and a stable frame rate of 15~25fps, thus achieving a balance between accuracy and speed.

[0076] 3. Low Frame Rate Mode Adjustment: The input resolution is reduced to 65% of the high frame rate mode (e.g., when the optimized first feature map resolution is H / 64×W / 64, it is adjusted to H / 98×W / 98), further reducing the computational load (by approximately 58%); the original CSPDarknet-53 network is replaced with a lightweight YOLOv8n backbone network, and the number of residual blocks is reduced (from 23 to 16) and the number of convolutional kernels is adjusted (from 3×3 to 1×1+3×3 hybrid convolution), reducing the number of model parameters (by approximately 60%), and significantly improving inference speed; the ACmix module of the P5 branch is disabled. The P5 branch corresponds to ultra-deep semantic features and has a low contribution to small object detection. Only the small object detection function of the P3 branch is retained. The P3 branch corresponds to shallow detail features and has a high contribution to small object detection, so the small object detection capability is prioritized. This mode is suitable for scenarios with high edge computing node load, such as mining areas with more than 10 mining cards online at the same time and insufficient inference speed. It improves inference speed by more than 40%, with a frame rate of ≥15fps and small target detection mAP@0.5≥0.85, avoiding remote control decision delays caused by low frame rate.

[0077] Step S414: Smooth model parameters (when switching frame rate modes).

[0078] When the frame rate mode changes, such as from high frame rate mode to medium frame rate mode, the FR-MSM module smooths model parameters, such as the weights of the ACmix attention head and the bounding box regression parameters of the detection head, using the Exponential Moving Average (EMA) algorithm. This prevents abrupt changes in model parameters caused by the mode switch, which could lead to jitter in the detection results, such as a shift in the position of small targets exceeding 3 pixels. The formula for the EMA algorithm is: ; in, Let be the EMA smoothing parameter at time t. The EMA smoothing parameters at time t-1 (model parameters before switching) are given. These are the optimal parameters for the current mode at time t (the parameters of the model after the switch). The attenuation coefficient ( The value of is determined by the "parameter smoothness - detection stability" curve to ensure smooth parameter transition and no obvious jitter in the detection results. Through this algorithm, the model parameters achieve a linear transition during switching, and the offset of the small target position is controlled within 3 pixels. This avoids the jitter problem caused by sudden parameter changes, such as "the small target is at (x1, y1) in the previous frame and jumps to (x1+6, y1+6) in the next frame", ensuring that remote operators can stably judge the position of the small target.

[0079] Step S42: Perform feature fusion and detection inference, and output obstacle detection results.

[0080] The adjusted and optimized first feature map and the optimized second feature map are input into the YOLOv8 feature fusion layer (PANet structure). Through bidirectional feature fusion from top to bottom and bottom to top, the detailed features of the optimized first feature map and the semantic features of the optimized second feature map are fused to obtain a fused feature map (1024 channels, resolution H / 64×W / 64). Then, the fused feature map is input into the YOLOv8 detection head. The detection head calculates the obstacle category probability (such as gravel, potholes, large obstacles) through the classification branch (using the cross-entropy loss function), and calculates the obstacle bounding box coordinates and confidence (reflecting the reliability of the detection results) through the regression branch (using the CIoU loss function). Finally, the obstacle detection result containing the obstacle location, size, category, and confidence is output.

[0081] For example, when a remotely controlled emergency rescue vehicle travels to the ruins area after an earthquake, the FR-MSM module detects that the frame rate drops to 12fps due to the edge computing nodes processing multiple video streams simultaneously. It automatically switches to low frame rate mode, which is retained by the lightweight backbone network and P3 branch. It can still accurately output the location (error < 2 pixels), size (error < 1 pixel), and category (confidence > 0.9) of rubble (small targets) in the ruins, providing accurate basis for remote control avoidance.

[0082] Step S5: Combine the obstacle detection results with the transmission rate calculation, classify them according to hazard level, and transmit them via a UDP / TCP hybrid protocol to generate control commands that are fed back to the vehicle controller. Please refer to [link / reference]. Figure 3 .

[0083] This step addresses the needs of bandwidth fluctuation adaptation and secure operation in remote control scenarios. It calls upon the obstacle detection results generated in step S4, combines them with transmission bitrate calculation to formulate a hierarchical transmission strategy, transmits key data through a hybrid protocol and generates control commands, thereby solving the shortcomings of traditional fixed bitrate transmission and ensuring the timeliness and accuracy of remote control.

[0084] Specifically, the steps for graded transmission, hazard level assessment, and instruction generation are as follows: Step S51: Calculate the average transmission bit rate and determine the bit rate level.

[0085] The obstacle detection results output from step S4 are input into the rate-adaptive feature splitting transmission module (AR-FSTM module). Simultaneously, the AR-FSTM module monitors the transmission rate between the remotely controlled vehicle and the remote cockpit in real time, and formulates a tiered transmission strategy based on the transmission rate. Specifically, the process for rate calculation and tier determination is as follows: Step S511: Calculate the average transmission code rate .

[0086] Real-time monitoring of transmission bitrate using a sliding window algorithm: Select 10 consecutive frames of feature data as a sliding window, and count the number of bytes in each frame of feature data within the window. With transmission time (Unit: s), the formula is: ; Where 8 is the byte-to-bit conversion coefficient (1 byte = 8 bits). The unit is bps (bits per second). This formula calculates the average transmission bitrate by accumulating the total number of bits across 10 frames of data and the total transmission time. This avoids bitrate judgment errors caused by fluctuations in the data volume of a single frame (such as a frame containing multiple large objects causing a surge in data volume), ensuring... It can accurately reflect the bandwidth status of the current communication link (such as the decrease in bit rate caused by 4G signal fluctuations in the mining area).

[0087] Step S512: Determine the bitrate level.

[0088] Based on average transmission rate The current bitrate level is determined by the threshold for classifying bitrate levels based on the bandwidth requirements of the remote control scenario (e.g., a minimum bitrate of 0.5 Mbps is required for small target feature transmission) and the bandwidth limit of the communication link (e.g., a maximum bitrate of 2 Mbps for satellite communication). The specific determination rules are as follows: When When the bit rate exceeds the high bit rate threshold (e.g., 2Mbps), it is determined to be a high bit rate mode (sufficient bandwidth to transmit complete feature data); when When the bit rate falls between the high bit rate threshold and the low bit rate threshold (e.g., 1 Mbps), it is determined to be a medium bit rate mode (medium bandwidth, requiring selective transmission of characteristic data); when When the bit rate is below the low bit rate threshold, it is determined to be a low bit rate mode (bandwidth is limited, and only core feature data is transmitted).

[0089] Step S52: Develop a hierarchical transmission strategy based on bit rate levels.

[0090] For different bitrate modes, the AR-FSTM module combines small target information from obstacle detection results to formulate a hierarchical transmission strategy: 1. High-bitrate mode transmission strategy: Transmits complete P3 / P4 / P5 three-branch feature data (P3 branch corresponds to small target features, P4 branch to medium target features, and P5 branch to large target features) and obstacle detection results (including the position, size, category, and confidence level of all obstacles), ensuring that the detection accuracy of large and small targets is not affected by transmission. This mode is suitable for scenarios with sufficient communication link bandwidth (such as 5G private network coverage areas in ports), with 100% feature data transmission integrity, and small target detection accuracy consistent with local inference accuracy (mAP@0.5≥0.92).

[0091] 2. Medium Bitrate Mode Transmission Strategy: Only feature data from branches P3 (small targets) and P4 (medium targets) are transmitted. For branch P5 (large targets), feature distillation technology is used to extract key semantic information. A pre-trained distillation model compresses the high-dimensional features (2048 channels) of branch P5 into low-dimensional semantic vectors (512 channels), reducing data volume by more than 50% while retaining the category and approximate location information of large targets (error < 5 pixels). In obstacle detection results, detailed information for small and medium targets is transmitted first, while only the category and confidence level of large targets are transmitted. This mode is suitable for scenarios with moderate communication link bandwidth (such as 4G signals in mining edge areas), reducing data volume by 30% while ensuring that the accuracy decrease for small target detection is < 3% (mAP@0.5≥0.89).

[0092] 3. Low Bitrate Mode Transmission Strategy: Only core feature data of small targets in the P3 branch (such as edge texture and position features of small targets, 256 channels) are transmitted. A feature reconstruction algorithm recovers some P4 branch feature information from the P3 branch features. Based on a pre-trained Generative Adversarial Network (GAN), approximate features of the P4 branch are generated using the P3 branch features as input (similarity ≥ 85% with the original P4 branch features), reducing the transmission volume of P4 branch features. Feature completion functionality is enabled on edge computing nodes, and missing P5 branch features are completed at the remote control terminal using a generative model, avoiding misjudgment of large targets due to missing P5 branch features. The obstacle detection results only transmit detailed information about small targets (position, size, category, confidence level) and a rough location of large targets (error < 10 pixels). This mode is suitable for scenarios with limited communication link bandwidth (such as satellite communication in remote areas during emergency rescue), reducing data volume by more than 60%, controlling transmission latency to within 100ms, and reducing small target detection accuracy by < 5% (mAP@0.5 ≥ 0.87).

[0093] Step S53: Assess the hazard level of small targets and transmit data via a hybrid UDP / TCP protocol.

[0094] Building upon tiered transmission, the AR-FSTM module assesses the hazard level of small targets based on obstacle detection results and constructs a priority queue to ensure the priority transmission of critical data from high-hazard-level small targets. Specifically, the process of hazard level assessment and hybrid protocol transmission is as follows: Step S531: Calculate the hazard level of the small target.

[0095] The formula for calculating the hazard level S of a small target is: ; This formula is original. Existing alternatives that can achieve similar functions include: the TTC (Time To Collision) model in autonomous driving. Collision time is calculated using distance and relative velocity, and a collision probability model is used. The collision probability is calculated based on distance d, velocity v, and acceleration a, and semantic priority (e.g., fixed priority of pedestrians > vehicles > static obstacles). Traditional methods rarely consider "size" as an independent hazard factor (larger obstacles are generally considered more dangerous). However, this invention addresses the problem of small targets being easily missed in remote control scenarios by proposing a design logic of "smaller targets are more dangerous." ( For small target pixel area, The image pixel area represents the risk level; the smaller the size, the larger the factor value, and the higher the risk level. A distance factor is also introduced. ( (where k is the attenuation coefficient, and k is the distance between the small target and the vehicle; the closer the distance, the larger the factor value) and the speed factor. (v is the speed of the small target relative to the vehicle,) (The maximum relative velocity threshold; the higher the relative velocity, the larger the factor value), weighted... ( The distance factor has the highest weight, which is consistent with the security logic of "prioritizing close-range threats". The hazard level S (with a value range of 0 to 1, the larger the S is, the higher the hazard level) is obtained by weighted fusion through training with accident statistics in remote control scenarios.

[0096] Step S532: Build a priority queue and transfer data.

[0097] Construct a priority queue based on the danger level S: The smaller goals are classified as the highest priority. The classification is divided into medium priority. Data is categorized into low-priority segments and transmitted using a hybrid UDP+TCP protocol: High-priority small target key features (such as location (x, y) and danger level S) are transmitted using TCP. TCP employs a three-way handshake and retransmission mechanism to ensure no packet loss, preventing operational errors caused by data loss of high-risk small targets. Medium- and low-priority feature data (such as background area features and approximate location of large targets) are transmitted using UDP. UDP requires no connection establishment, resulting in low transmission latency and improved overall transmission speed. This hybrid protocol transmission strategy keeps the packet loss rate below 1%, ensuring 100% reliability for high-risk small target data transmission, thus solving the problems of packet loss with traditional single UDP protocols or high latency with single TCP protocols.

[0098] Step S54: Generate control commands and send them back to the vehicle controller.

[0099] The remote cockpit receives feature data and obstacle detection results transmitted by the AR-FSTM module and displays them through a visual interface: marking the location of small obstacle targets in the road image (highest priority small targets are marked with red rectangles, medium priority targets with yellow rectangles, and low priority targets with blue rectangles), their size (the side length of the rectangle corresponds to the actual size), category (labeled with text such as "gravel" or "pothole"), and hazard level (labeled in numerical form next to the rectangle), allowing remote operators to intuitively judge road conditions. At the same time, the remote cockpit's control decision system automatically generates preliminary control commands based on the obstacle detection results (such as "decelerate to 5 km / h" or "avoid to the left by 0.5 meters"). The operator can adjust the commands according to the visual information and then feed them back to the high-performance automotive-grade SOC controller of the vehicle perception terminal through the data transmission module. After receiving the control commands, the SOC controller converts them into control signals for the vehicle actuators (such as accelerator, brake, and steering) to achieve obstacle avoidance control of the vehicle and complete one detection cycle.

[0100] For example, when the remotely controlled mining truck travels to the slope area of ​​the mining area, the AR-FSTM module detects that the transmission bit rate has dropped to 0.8Mbps (low bit rate mode), and only transmits the core features of the gravel (small target) of the P3 branch. The hazard level formula is used to calculate the gravel at the slope. (Highest priority) The location and size information of the rock are transmitted using the TCP protocol; the remote cockpit visual interface clearly marks the rock, and the operator generates a control command of "avoid to the right by 0.3 meters", which is fed back to the mining truck SOC controller. The mining truck successfully avoids the rock and avoids tire damage.

[0101] The generated control commands can directly drive the on-board actuators to complete obstacle avoidance operations, ensuring the driving safety of remotely controlled vehicles in complex scenarios. At the same time, the flow of variables throughout the entire process forms a complete closed loop. From obstacle detection results to hierarchical transmission and command generation, the output of each link becomes the input of the next link, ensuring the continuity and reliability of the process.

[0102] Example 2: This embodiment, based on Embodiment 1, provides a remote-controlled vehicle road obstacle detection system, such as... Figure 4 As shown, it includes: Image data acquisition and packaging module: used to acquire road images through a high-definition camera and generate image data packets by attaching frame rate metadata; Feature extraction and denoising module: It is used to input image data packets into the improved YOLOv8 backbone network, extract the first feature map and the second feature map, and perform dynamic weighted fusion through the dual-branch residual attention denoising module to output the denoised feature map; Feature optimization module: It is used to input the denoised feature map into the spatial pyramid pooling-fast Fourier convolution module to perform downsampling, and then input it into the dual-coordinate attention anti-noise feature extraction module to perform anti-noise optimization and long-range dependency enhancement, and output the optimized first feature map and second feature map; The adaptive detection and inference module is used to input the optimized first feature map and second feature map into the YOLOv8 feature fusion layer and the detection head, and adaptively adjust the model structure through the frame rate-aware multi-mode switching module to output obstacle detection results. Hierarchical transmission and command generation module: It is used to calculate the transmission bit rate and formulate hierarchical transmission strategies through the feature-based transmission module with adaptive bit rate, assess the danger level of small targets and transmit data through UDP / TCP hybrid protocol, and generate control commands to be fed back to the vehicle controller.

Claims

1. A method for detecting road obstacles in a remotely controlled vehicle, characterized in that, The method includes: S1: Capture road images using a high-definition camera and generate image data packets by attaching frame rate metadata; S2: Input the image data packet into the improved YOLOv8 backbone network, extract the first feature map and the second feature map, and then process the first feature map and the second feature map through a two-branch residual attention denoising process to output their respective denoised feature maps after dynamic weighted fusion; wherein, the improved YOLOv8 backbone network uses CSPDarknet-53 as the core component, and the first feature map and the second feature map are the feature tensors output by the third and fourth groups of CSP residual modules in the network, respectively; the two branches of the two-branch residual attention denoising process are a spatial variation noise suppression branch for suppressing noise and a contrast enhancement branch for illumination compensation; S3: The denoised feature map is downsampled and optimized using dual-coordinate attention noise reduction. Combined with long-range dependency enhancement of shallow and deep features, optimized first and second feature maps are output. The downsampling generates deep and ultra-deep feature maps. The dual-coordinate attention noise reduction optimization includes constructing attention weight maps in the x and y axes respectively and multiplying them pixel-by-pixel with the deep feature map. Noise-aware downsampling is performed in conjunction with local noise intensity. The noise-aware downsampling includes: median filtering preprocessing when the local noise intensity is higher than the downsampling noise threshold, and direct average pooling operation when the local noise intensity is lower than the downsampling noise threshold. This noise-aware downsampled feature map is defined as the optimized first feature map. The long-range dependency enhancement includes transferring shallow detail information from the first feature map to the ultra-deep feature map through a cross-attention mechanism to generate an enhanced ultra-deep feature map, which is then defined as the optimized second feature map. S4: Input the optimized first feature map and second feature map into the YOLOv8 feature fusion layer and the detection head, adaptively adjust the model structure according to the current frame rate level, and output obstacle detection results including position, size, category and confidence; wherein, the adaptive adjustment of the model structure according to the current frame rate level includes: calculating the average frame rate to divide the frame rate level, and adjusting the input resolution, the number of attention module activations and the number of candidate boxes accordingly, and smoothing the model parameters through the exponential moving average algorithm when switching frame rate modes; S5: Combines obstacle detection results with transmission rate calculation, classifies them according to hazard level, and transmits them via UDP / TCP hybrid protocol to generate control commands that are fed back to the vehicle controller.

2. The method for detecting road obstacles in a remotely controlled vehicle according to claim 1, characterized in that, The steps for generating the image data packet include: S11: Collects road image data through high-definition cameras; S12: Collect frame rate data via a performance counter; S13: Encapsulate the road image data and frame rate data into an image data packet; the structure of the encapsulated image data packet consists of a frame header, image data, frame rate metadata, and a checksum.

3. The method for detecting road obstacles in a remotely controlled vehicle according to claim 1, characterized in that, The steps of extracting the first feature map and the second feature map, and then performing dual-branch residual attention denoising processing to output a dynamically weighted fused denoised feature map include: S21: Parse the image data packet to obtain the pixel tensor and frame rate metadata to be processed; parsing includes confirming the data type through the data packet type identifier in the frame header, extracting the image data and frame rate metadata according to the data length field, and converting the image data into a three-dimensional pixel tensor; S22: Based on the improved YOLOv8 backbone network, extract the first feature map and the second feature map; the improved YOLOv8 backbone network is based on the original YOLOv8 architecture; S23: Input the first feature map and the second feature map into the dual-branch residual attention denoising module, perform dynamic weighted fusion, and output the denoised feature map; the dual-branch residual attention denoising module is based on dynamic weighted fusion of noise intensity.

4. The method for detecting road obstacles in a remotely controlled vehicle according to claim 3, characterized in that, The steps for extracting the first feature map and the second feature map based on the improved YOLOv8 backbone network include: S221: Adjust the number of channels of the tensor of the pixel to be processed through a 1×1 convolutional layer, and at the same time perform downsampling with a stride of 2 through a 3×3 convolutional layer to obtain the initial feature tensor; S222: The initial feature tensor is sequentially input into 5 sets of CSP residual modules for progressive processing according to the rule of doubling the number of channels and halving the resolution, and the output is an intermediate feature tensor sequence containing 5 stage feature tensors; the CSP residual module superimposes the input feature map with the convolutional feature map through residual connection; S223: From the intermediate feature tensor sequence of the steps, extract the feature tensor after processing by the third group of CSP residual modules as the first feature map, and extract the feature tensor after processing by the fourth group of CSP residual modules as the second feature map.

5. The method for detecting road obstacles in a remotely controlled vehicle according to claim 3, characterized in that, The step of inputting the first feature map and the second feature map into the dual-branch residual attention denoising module and performing dynamic weighted fusion includes: S231: Perform spatial variation noise suppression branch processing to generate a denoised branch feature map; the spatial variation noise suppression branch adopts an improved HDRAB, which captures noise distributions at different scales through 3-level dilated convolution and introduces residual connections, which directly superimpose the input first feature map or second feature map with the feature map after the 3-level dilated convolution processing; S232: Perform contrast enhancement branch processing to generate an enhanced branch feature map; the contrast enhancement branch is based on Retinex theory, decomposes the feature map into illumination component and reflection component, and performs illumination compensation on the reflection component of low illumination area. S233: Calculate noise intensity; S234: Generate an attention fusion weight matrix based on noise intensity, dynamically weight and fuse the denoised branch feature map and the enhanced branch feature map, and output the denoised feature map.

6. The method for detecting road obstacles in a remotely controlled vehicle according to claim 1, characterized in that, The step of downsampling and dual-coordinate attention noise reduction optimization of the denoised feature map includes: S31: Input the denoised feature map into the spatial pyramid pooling-fast Fourier convolution module to perform downsampling operation and generate deep feature maps and ultra-deep feature maps; the spatial pyramid pooling-fast Fourier convolution module combines spatial pyramid pooling and fast Fourier convolution, first performs preliminary downsampling, then performs spatial pyramid pooling through multi-scale pooling kernels, and finally decomposes low-frequency components and high-frequency components, converts the low-frequency components and high-frequency components back to the spatial domain through inverse fast Fourier transform and concatenates them to obtain deep feature maps and ultra-deep feature maps; S32: Input the deep feature map and the ultra-deep feature map into the dual-coordinate attention noise-resistant feature extraction module, perform noise-resistant optimization and long-range dependency enhancement, and output the optimized first feature map and the optimized second feature map.

7. The method for detecting road obstacles in a remotely controlled vehicle according to claim 6, characterized in that, The step of inputting the deep feature map and the ultra-deep feature map into the dual-coordinate attention noise reduction feature extraction module includes: S321: Execute a parallel coordinate attention mechanism to generate a coordinate attention feature map; the parallel coordinate attention mechanism constructs attention weight maps in the x-axis and y-axis directions respectively, and multiplies the deep feature map with the attention weight map pixel by pixel; S322: Perform noise-aware downsampling optimization to generate a noise-aware downsampling feature map; the noise-aware downsampling optimization first calculates the local noise intensity through the noise detection window, selects median filtering preprocessing or direct average pooling according to the local noise intensity, and adjusts the channel segmentation ratio of the feature tensor to 3:2, where 3 / 5 of the channels are used to retain edge texture features and 2 / 5 of the channels are used to retain semantic features; S323: Perform long-range dependency enhancement to generate an enhanced ultra-deep feature map; the long-range dependency enhancement adjusts the first feature map to the same resolution as the ultra-deep feature map through multi-level attention propagation, and then transmits shallow detail information to deep features through a cross-attention mechanism; S324: Define the noise-aware downsampled feature map as the optimized first feature map and the enhanced ultra-deep feature map as the optimized second feature map.

8. The method for detecting road obstacles in a remotely controlled vehicle according to claim 1, characterized in that, The steps for outputting the obstacle detection results include: S41: Determine the frame rate level based on frame rate metadata and adaptively adjust the YOLOv8 model structure; the adaptive adjustment is performed by the frame rate-aware multi-mode switching module. S42: Perform feature fusion and detection inference, and output obstacle detection results; the feature fusion is performed through the YOLOv8 feature fusion layer to perform bidirectional feature fusion, and the detection inference calculates the obstacle category probability through the classification branch and calculates the obstacle bounding box coordinates and confidence through the regression branch.

9. The method for detecting road obstacles in a remotely controlled vehicle according to claim 1, characterized in that, The step of generating control commands includes: S51: Calculate the average transmission bit rate and determine the bit rate level; The calculation of the average transmission bit rate is performed by the bit rate adaptive feature split transmission module through a sliding window algorithm, and high, medium and low bit rate levels are divided according to the average transmission bit rate. S52: Based on the bit rate level, formulate a hierarchical transmission strategy; the hierarchical transmission strategy selects to transmit complete feature data, selectively transmit feature data, or transmit only core feature data according to different bit rate levels; S53: Assess the hazard level of small targets and transmit data via a hybrid UDP / TCP protocol; the hazard level of small targets is calculated based on a weighted average of size factor, distance factor, and velocity factor, with the highest priority data transmitted using the TCP protocol and the medium and low priority data transmitted using the UDP protocol. S54: Generate control commands and feed them back to the vehicle controller; the generated control commands include remote cockpit visualization of obstacle information, automatic generation of preliminary control commands, which are then adjusted and fed back to the vehicle controller to be converted into actuator control signals.

10. A remote-controlled vehicle road obstacle detection system, used to implement the remote-controlled vehicle road obstacle detection method according to any one of claims 1-9, characterized in that, The system includes: Image data acquisition and packaging module: used to acquire road images through a high-definition camera and generate image data packets by attaching frame rate metadata; Feature extraction and denoising module: This module inputs image data packets into the improved YOLOv8 backbone network, extracts a first feature map and a second feature map, and then performs dynamic weighted fusion of the first and second feature maps through a dual-branch residual attention denoising module to output their respective denoised feature maps. The improved YOLOv8 backbone network uses CSPDarknet-53 as its core component, and the first and second feature maps are the feature tensors output by the third and fourth groups of CSP residual modules in this network, respectively. The two branches of the dual-branch residual attention denoising process are a spatial variation noise suppression branch for noise reduction and a contrast enhancement branch for illumination compensation. Feature optimization module: This module inputs the denoised feature map into the spatial pyramid pooling-fast Fourier convolution module for downsampling, then inputs it into the dual-coordinate attention-based noise reduction feature extraction module for noise reduction optimization and long-range dependency enhancement, outputting optimized first and second feature maps. The downsampling generates deep and ultra-deep feature maps. The dual-coordinate attention-based noise reduction optimization includes constructing attention weight maps along the x and y axes and multiplying them pixel-by-pixel with the deep feature map, combining this with local noise intensity for noise-aware downsampling. This noise-aware downsampling includes: median filtering preprocessing when the local noise intensity is higher than the downsampling noise threshold, and direct average pooling when the local noise intensity is lower than the downsampling noise threshold. This noise-aware downsampling feature map is defined as the optimized first feature map. The long-range dependency enhancement includes transferring shallow detail information from the first feature map to the ultra-deep feature map through a cross-attention mechanism to generate an enhanced ultra-deep feature map, which is then defined as the optimized second feature map. The adaptive detection and inference module is used to input the optimized first feature map and second feature map into the YOLOv8 feature fusion layer and the detection head, and adaptively adjust the model structure through the frame rate-aware multi-mode switching module to output obstacle detection results. The adaptive adjustment of the model structure according to the current frame rate level includes: calculating the average frame rate to divide the frame rate level, and adjusting the input resolution, the number of attention module activations and the number of candidate boxes accordingly, and smoothing the model parameters through the exponential moving average algorithm when switching frame rate modes. Hierarchical transmission and command generation module: It is used to calculate the transmission bit rate and formulate hierarchical transmission strategies through the feature-based transmission module with adaptive bit rate, assess the danger level of small targets and transmit data through UDP / TCP hybrid protocol, and generate control commands to be fed back to the vehicle controller.

Images